Research Reports

Detection of Large-Scale Item Preknowledge in Computerized Adaptive Testing via Kullback–Leibler Divergence (RR 12-01)

Detection of test collusion is a new research direction in the area of test security. Test collusion may be described as large-scale sharing of test materials, including answers to test questions (items). Test collusion for high-stakes testing programs can affect the scoring of the exam because of the potentially large number of test takers involved. Therefore, identifying such test takers in order to withdraw their responses from the data is an important task. Current methods of detecting test collusion are based on statistics also used in answer-copying detection. In computerized adaptive testing, these methods lose power because the items presented are not the same for all test takers. This report addresses this problem by introducing a new approach that searches through various partitions of the test and analyzes differences between corresponding distributions of test-taker ability. The suggested approach was found to be effective in detecting groups of test takers with item preknowledge, meaning those with access to a (possibly unknown) subset (or subsets) of items prior to the exam.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.