Why LSAT Scores Should Not Be Used to Label Law Schools and Their Applicants
DANIEL O. BERNSTINE
President, LSAC
President, LSAC
A report recently released by Law School Transparency (LST) has gained headlines by claiming that some ABA-approved law schools have been intentionally admitting “high risk” students who, based on their LSAT scores, do not have a reasonable chance of passing the bar. As explained below, this claim and others made by the LST study are based on misunderstandings of the LSAT.
The LSAT is a valid measure of certain cognitive skills that are important for success in law school. However, proper use of the test does not include using score ranges to label law schools and their students as to their potential for successful bar passage.
In addition to the fact that the recommendations in the LST report run counter to LSAC’s test-use guidelines, we are concerned about the methodological errors upon which the report is based, and the misleading media coverage that this flawed report has generated. Consider the following three claims made by the LST report that are demonstrably false, and yet are being perpetuated by both major and minor media outlets.
False Claim #1: LSAT scores can be used to assign bar passage risk.
The LST study itself provides examples that contradict this claim. Tables from the study indicate that students labeled “high risk” (i.e., those with LSAT scores in the 147–149 range) who attended one law school had a first-time pass rate of 57 percent, while comparable students from another school had a pass rate of 23 percent. Clearly, many factors significantly affect bar passage rates above and beyond LSAT scores. The assertion that LSAT scores alone measure comparability is patently wrong.
False Claim #2: LSAT scores can be used to delineate risk categories.
According to the risk framework adopted by the LST study, two students with LSAT scores of 149 and 150 would be labeled “high risk” and “modest risk,” respectively, even though a score difference of one point could very likely be attributable to measurement error alone rather than a meaningful difference in test-taker ability. Moreover, if the student with an LSAT score of 149 fell at the 25th percentile at their law school, that institution would be labeled “high risk” or “serious risk,” whereas if the student with an LSAT score of 150 fell at the same percentile at their law school, that institution would be labeled “modest risk.” LSAC has long cautioned against drawing such fine distinctions in LSAT scores.
False Claim #3: A study based on 25-year-old data can be used to assess current bar passage risk.
The LSAC National Longitudinal Bar Passage Study, cited as one source for the LST report, was based on scores from a previous version of the LSAT. The results from this study cannot be applied directly to current LSAT scores, as claimed by LST. The LSAC study did state that “from the perspective of entry to the profession, the eventual pass rate is a far more important outcome than first-time pass rate.” This statement is as true today as it was 25 years ago. As noted by LST, the LSAC study participants who scored below the (then) average LSAT score had an eventual bar passage rate of over 90 percent.
As noted above, the LSAT measures a limited set of skills that are important for success in law school. The LSAT does not measure all attributes that can predict the probability of eventual bar passage or professional success after graduation. As stated in LSAC’s Cautionary Policies Concerning LSAT Scores and Related Services, “The LSAT should be used as only one of several criteria for evaluation and should not be given undue weight solely because its use is convenient. Those who set admission policies and criteria should always keep in mind the fact that the LSAT does not measure every discipline-related skill necessary for academic work, nor does it measure other factors important to academic success.” An organization such as LST, which purports to advocate transparency, would do well to acknowledge this advice, and follow it in its reporting.