Fit to Testlet Models and Differential Testlet Functioning (RR 12-07)
by Cees A. W. Glas, University of Twente, Enschede, The Netherlands
Executive Summary
A set of questions (i.e., items) grouped around a common stimulus is often referred
to as a testlet. Law School Admission Test (LSAT) Reading Comprehension and
Analytical Reasoning items are presented in a testlet format. Recent research has
proposed two potential models for evaluating whether testlet response behavior can be
predicted on the basis of various testlet features. This report presents a general method
for evaluating the suitability not only of these models, but of the original model from
which they are derived.
Differential item functioning and differential testlet functioning pertain to differences
in responses to specific items and testlets that can be attributed not only to the
proficiency assessed, but also to irrelevant background variables such as, for instance,
gender or race/ethnicity. The model evaluation method presented here is further applied
to the detection of differential item and testlet functioning, as well as to the evaluation of
other statistical assumptions required by the models. The method was applied to two
sets of LSAT data, and no serious model violations were detected.