Research Reports

An Overview of Research on the Testlet Effect: Associated Features, Implications for Test Assembly, and the Impact of Model Choice on Ability Estimates (RR 13-03)

A mathematical model called item response theory is often applied to high-stakes tests to estimate test-taker ability level and to determine the characteristics of test questions (i.e., items). Often, these tests contain subsets of items (testlets) grouped around a common stimulus. This grouping often leads to items within one testlet being more strongly correlated among themselves than among items from other testlets, which can result in moderate to strong testlet effects.

A series of research projects was undertaken to investigate the theoretical and practical implications of the testlet effect for high-stakes tests such as the Law School Admission Test. These projects explored areas such as the development of a testlet response model to account for the testlet effect and the development of model fit statistics to accompany the model. The model was also applied to investigate the relationship between the stimulus features and the statistics used to describe individual test items, as well as the impact of the testlet effect on the assembly of test forms.

The current paper begins by summarizing findings across the series of research projects and goes on to investigate the impact of model choice on test assembly and estimates of test-taker ability. Finally, important topics that need to be addressed by future studies are discussed.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.