Research Reports

Statistical Algorithms for Detection of Test Collusion (RR 11-03)

The development of statistical methods for detecting test collusion is a new research direction in the area of test security. Test collusion may be described as large-scale sharing of test materials or answers to test questions. The source of the test materials could be a teacher, a test-preparation company, the Internet, or test takers communicating on the day of the exam. The danger of test collusion for high-stakes testing programs is that it can seriously affect the scoring of the exam because of the potentially large number of test takers involved. This is a serious concern for the test users (law schools, universities, companies, government organizations, etc.), who will be given invalid scores for test takers involved in test collusion. Therefore, identifying such test takers in order to remove their responses from the data is an important task.

Test collusion often influences the test performance or speed of involved test takers for the affected portion of the test. However, the portion of the test where the collusion took place is often unknown, and searching through all possible portions will decrease the detection power or increase the false-positive rate of a statistical test. This report introduces an algorithm that resolves this problem by working in two stages. In the first stage, test centers with potential collusion are identified as aberrant. In the second stage, potentially aberrant test takers from the aberrant test centers are identified. A simulation study demonstrates the advantages of using this algorithm for the Law School Admission Test (a case of a partially stolen section of the LSAT was simulated). However, the algorithm is general and can be applied to other testing formats, including computer-based testing (CBT), multiple-stage testing (MST), and computerized adaptive testing (CAT).

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.