LSAC Resources - Research

Sequencing an Adaptive Test Battery (RR 08-05)

by Wim J. van der Linden, University of Twente, Enschede, The Netherlands

Executive Summary

A test battery is a group of subtests intended for administration to the same group of test takers. Test batteries usually have severe time constraints on their subtests, but the abilities they measure are highly correlated. Hence, such batteries stand to profit substantially from computerized adaptive testing (CAT), wherein test questions (items) are selected for administration to a test taker based on their performance on previous items, with the intent of tailoring the difficulty level of the test to the ability level of the test taker. In this research, the idea of adaptation was extended further to include the sequencing of the subtests during the administration of the battery. While traditionally the sequence is arbitrarily fixed, this research suggests adapting it to the performances of the individual test takers. The optimal subtest to be administered first in the battery is chosen based on statistical information that is available about the test items and test-taker population prior to the test administration. The second subtest is chosen to be optimal given the test taker's responses within the first subtest, and so on. For the results to be realistic, content constraints are also met in the item-selection process.

The efficiency of the framework was explored in a computer simulation study with a battery consisting of short adaptive subtests for the three sections of the LSAT. The subtests were simulated from a previous item pool from the Law School Admission Council with a selection from its actual content constraints imposed on the individual subtests. The study confirmed the predicted gain in accuracy for the test battery, especially for conditions with higher correlations among the abilities measured by the subtests. In fact, even for subtests as short as five items, the observed error for the ability estimates quickly approximated the ideal.

• Full Research Report (PDF)