Research Reports

Distributions of Kullback–Leibler Divergence and Its Application for the LSAT (RR 09-02)

It is crucial for any testing organization to be able to recognize inconsistencies in test-taker performance across multiple standardized test administrations or across sections within a single administration. One method of recognizing inconsistent performance is to construct multiple posterior distributions and to compare the divergence between pairs of such distributions. (Note: In the context of a test taker's performance, a posterior distribution summarizes what we know about the probability associated with certain levels of performance by the test taker.)

Comparing posteriors takes into account all information available from the responses and allows consideration of various partitions of the test taker's responses to test questions (items). Practical partitions include scored versus unscored items, hard versus easy items, unexposed versus exposed items, uncompromised versus compromised items, and items of one type versus items of another type. A large divergence between posteriors indicates a significant change in a test taker's performance. Such changes could be an indication of answer copying, item pre-knowledge, or test-taker pre-identification of the unscored section.

This paper evaluates the use of Kullback–Leibler divergence (KLD) to compare posterior distributions in the context of the Law School Admission Test. The statistical characteristics of KLD are presented and evaluated. KLD has been applied for this purpose in the context of magnetic resonance imaging, human gene analysis, stochastic complexity, and sample size selection. It is concluded that the properties of KLD support its use in the comparison of posterior distributions to identify inconsistent test-taker response cases.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.