Research Reports

A Hierarchical Framework for Modeling Speed and Accuracy on Test Items (RR 05-02)

In the analysis of data for the Law School Admission Test (LSAT) and other similar standardized tests, a mathematical model called item response theory (IRT) is commonly used to estimate both the characteristics of the test questions (items) and the ability level of the test takers. Such analyses are based on the test takers’ correct and incorrect responses to the test items. When a test is administered in a computerized mode, the capability of recording the amount of time a test taker has spent on each item provides us with additional information about the test-taking experience of individuals as well as the characteristics of items.

The practical goal of this paper is to use response times on test items as an additional source of information in estimating the abilities of the test takers when the test is delivered in a computerized mode. It is only possible to use this additional source when we have a mathematical model that (i) relates the speed at which a test taker works to his/her ability (accuracy) on the test and (ii) separates the test taker’s speed on the test from the time intensities of the items.

A hierarchical framework of modeling is introduced that has two different levels, one for the individual test taker and one for the population of test takers. Each level includes two components: one to model speed and the other to model accuracy. At the level of the individual test taker, the framework models the test taker’s responses to items (correct or incorrect) and the time he or she spent on each item. Both component models have separate parameters for the item and person effects. At the second level, the framework has a model for the population of test takers that explains how the speed and accuracy of the test takers tend to be related. In addition, it has an item-domain model that relates the time intensities of the items to such features as their difficulties.

A method is applied to estimate all unknown parameters from the responses and times on the test items. A description of how the second level model can be used to predict a test taker's accuracy from his speed or the difficulties of the items from their time intensities is also included.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.