Research Reports

Optimal Detection of Aberrant Answer Changes (RR 16-02)

In standardized multiple-choice testing, test takers often change their answers for various reasons. The statistical analysis of answer changes (ACs) has uncovered multiple testing irregularities on large-scale assessments and is now routinely performed at some testing organizations. This report presents two new approaches to analyzing ACs at the individual test-taker level. The information about all previous answers is used only to partition the data into two disjoint subsets: responses where an AC occurred and responses where an AC did not occur. The objective was to develop an approach that would provide optimal statistical power. The two statistics presented are based on probabilistic aspects of the difference in performance between these subsets. Answer-changing behavior was simulated, where realistic distributions of wrong-to-right, wrong-to-wrong, and right-to-wrong ACs were achieved. Results of preliminary analyses were encouraging, as these proposed statistics outperformed other widely used methods.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.