Research Reports

Robust Automated Test Assembly for Testlet-Based Tests: An Illustration With the Analytical Reasoning Section of the LSAT (RR 13-02)

In many high-stakes tests, subsets of questions (i.e., items) grouped around a common stimulus are often utilized to increase testing efficiency. These subsets of items are commonly called testlets. Since responses to items belonging to the same testlet not only depend on the test taker’s ability, but also on the correct reading, understanding, and interpretation of the stimulus, the assumption that the responses to these items are independent of one another does not always hold.

A mathematical model called item response theory is often applied in automated test assembly (ATA) with testlets. Testlet response theory (TRT) models have been developed to deal with dependency among items within a testlet. This report addresses some of the questions that arise in the application of TRT models to ATA. Specifically, a robust ATA method is applied. The results obtained by this method, as well as the advantages it offers, are discussed. Finally, recommendations about the use of the new method are given.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.