An Application of the VM-Index for the LSAT (RR 10-02)
by Dmitry I. Belov
Executive Summary
From a test security perspective, answer copying is the issue of primary concern for high-stakes
paper-and-pencil tests. Answer-copying behavior often results in an unusual agreement
between the incorrect answers for a pair of test takers. Each pair consists of a subject (copier)
and a source, where the subject copies answers from the source. A common approach to
detecting unusual agreement between the incorrect answers of two test takers is the K-Index
probability. The K-Index has been used at the Educational Testing Service (ETS) for decades,
and its properties have been further explored in the literature. The major advantage of the K-Index
is its applicability to any multiple-choice response data. The major disadvantage of the
K-Index is a high Type II (i.e., false-negative) error rate when the source has a high score.
This report presents the VM-Index, a new statistic for detecting answer copying. The power
of the VM-Index lies in its evaluation of a matrix of distributions of response matches in the
variable (unscored) section of the LSAT. Note that the analysis of matches between variable
section responses is especially informative, since adjacent test takers will have different
variable sections. In a computational study, the VM-Index demonstrates a large decrease in
the Type II error rate and a small decrease in the Type I (i.e., false-positive) error rate when
compared with the K-Index.