LSAC Resources - Research

Fit to Testlet Models and Differential Testlet Functioning (RR 12-07)

by Cees A. W. Glas, University of Twente, Enschede, The Netherlands

Executive Summary

A set of questions (i.e., items) grouped around a common stimulus is often referred to as a testlet. Law School Admission Test (LSAT) Reading Comprehension and Analytical Reasoning items are presented in a testlet format. Recent research has proposed two potential models for evaluating whether testlet response behavior can be predicted on the basis of various testlet features. This report presents a general method for evaluating the suitability not only of these models, but of the original model from which they are derived.

Differential item functioning and differential testlet functioning pertain to differences in responses to specific items and testlets that can be attributed not only to the proficiency assessed, but also to irrelevant background variables such as, for instance, gender or race/ethnicity. The model evaluation method presented here is further applied to the detection of differential item and testlet functioning, as well as to the evaluation of other statistical assumptions required by the models. The method was applied to two sets of LSAT data, and no serious model violations were detected.

• Full Research Report (PDF)