LSAC Resources - Research

Conceptual Issues in Response-Time Modeling (RR 08-01)

by Wim J. van der Linden, University of Twente, Enschede, The Netherlands

Executive Summary

The literature on response-time (RT) modeling for test questions (items) has been quite unclear with regard to several underlying conceptual issues, such as (a) whether the RT distributions for correct and incorrect responses should be treated differently, (b) whether RTs should be modeled to be dependent on the difficulty of the item, (c) whether ability and speed are constrained by a tradeoff relationship, and (d) whether scoring rules based on total time and total number of items correct are equivalent.

In this report, conceptualizations for the following two dominant traditions of RT modeling are reviewed and analyzed: (a) distinct models for the distributions of RTs and the distributions of item responses, and (b) an integrated approach whereby RTs and item responses are incorporated into the same models. Six basic conclusions from the analyses are presented. One of the conclusions entails a fundamental equation relating RT to the speed of the test taker and the labor intensity of the test items, which should be the core of any RT model.

Also, the conclusions suggest a hierarchical modeling framework with distinct response and RT models as in the first tradition of RT modeling, but with the relationships between their parameters represented by a second level of modeling. Two examples of this hierarchical approach are discussed: one proposed by the current author in an earlier research project for the Law School Admission Council, and the other based on two models for the number of misreadings and speed on oral reading tests, proposed by another researcher. The flexibility of the framework in predicting correlations between RTs and responses was demonstrated empirically.

• Full Research Report (PDF)