A Bivariate Lognormal Response-Time Model for the Detection of Collusion Between Test Takers (RR 08-06)
by Wim J. van der Linden, University of Twente, Enschede, The Netherlands
Executive Summary
A classic form of collusion between test takers on multiple-choice tests is the signaling of question and answer numbers using a prearranged code (e.g., number of silent finger taps). For computer-based testing, more advanced forms of communication are possible through a local network or over the Internet when the test takers' computers have not been locked down properly. No doubt, new forms of collusion will arise soon, for instance, through the use of digital pens that automatically record handwritten responses and share them electronically with others.
The goal of this research is to detect collusion between test takers based on their response times (RTs) to test questions (items). RTs have the advantage of continuous rather than dichotomous information (i.e., correct or incorrect item scores) about test takers' behavior, and they are therefore more informative about the size of possible aberrances than are the responses to items. Also, their use is not restricted to any specific response format (i.e., multiple-choice versus open-ended or essay formats).
A model for pairs of test takers is developed that allows us to study the correlation between their RTs, and a statistical test of a hypothesis on this correlation coefficient is proposed. A few empirical examples are provided to show that the estimator and statistical test are able to distinguish between correlation due to actual collusion between test takers and spurious correlation due to variation in the time intensity of the items or agreement in speed between the two test takers. A discussion of how these two tools should be used in routine checks of test datasets for suspicious agreement between the RT patterns of different test takers is included.