Study Shows LSAT Score Increases for Candidates Who Use Free Khan Academy Prep Tools
By LSAC Research
The more time law school candidates spent preparing for the Law School Admission Test via Khan Academy’s free online LSAT preparation tools, and the more practice exams those candidates completed on the Khan Academy platform, the higher the candidates tended to score on the LSAT, according to a recently completed Law School Admission Council study that supports the value of this online resource for LSAT takers.
The researchers cautioned that Khan Academy practice activities could not yet be declared the sole cause of LSAT score increases, due to the “quasi-experimental” nature of the research. However, LSAC researchers Kimberly Dustman, Ann Gallagher, and Gregory Camilli wrote that “engagement with the Khan Academy platform is currently the best explanation for the LSAT score increases observed in this study.”
In looking at a period from June 2018 (when Khan Academy’s LSAT prep tool was launched) to July 2020, Dustman, Gallagher, and Camilli matched Khan Academy usage information with data from LSAT test takers who indicated they used Khan Academy for the test and gave LSAC permission to use their data for research purposes. This yielded a sample of 12,471 candidates whose usage of the Khan Academy and performance on the LSAT were analyzed. Roughly half of the students in the study sample did not complete a full Khan Academy practice exam, and the researchers said further research is needed to understand how to better engage students on the platform.
The researchers focused on two variables: the minutes candidates spent on Khan Academy practice activities, and the number of practice exams candidates took. They found a moderate, but positive and statistically significant, correlation between practice minutes and LSAT scores; in other words, the longer a candidate spent engaged with Khan Academy practice activities, the higher they tended to score on the LSAT.
Similarly, taking more practice LSAT exams, up to the maximum of 10 allowed on the platform, “generally resulted in increased LSAT scores relative to number of practice exams taken,” the researchers wrote. The study also found it unlikely that other influences, such as commercial LSAT prep courses, could have accounted for those increased scores.
Importantly, Dustman, Gallagher, and Camilli found that the positive effects of using the Khan Academy preparation tools were present across demographic subgroups. There is some evidence to suggest that Black, Hispanic, and female candidates tended to have the largest benefits, suggesting that engagement with the Khan Academy platform is of particular value for these historically underrepresented groups. However, this evidence is suggestive due to the small sample size for some subgroups; additional research is needed with a larger sample size to more deeply investigate.
For example, the study found that Black law school candidates who completed one or two practice tests saw an increase of 2.84 points, on average, in their LSAT score, while Black candidates who finished three or four practice LSATs saw an average increase of nearly 5 points. Hispanic and female candidates saw similar results, while increases for White and male candidates were present but slightly less pronounced.
In the overall study sample, candidates who completed one or two practice LSAT exams saw an average score increase of 1.59 points on the actual LSAT. That score boost increased to 3.58 points with three or four practice tests, 4.39 points with five or six practice tests, 5.59 points with seven or eight practice tests, and 7.26 points with nine or ten practice tests.
The Khan Academy platform allows candidates to create personalized practice plans that cater to each candidate's strengths and weaknesses. It includes interactive lessons, problem sets, timed practice exams, and instructional videos. In September of 2020, nearly 70,000 people used the Khan Academy platform in some capacity.
“Because the Khan LSAT platform is free of charge and easily accessible online, individuals preparing to take the LSAT may want to consider incorporating Khan Academy’s Official LSAT Prep activities,” the researchers wrote, adding that the Khan Academy platform “is likely a pragmatic alternative to commercial test prep courses and private tutoring.” In fact, the sample of Khan users was less likely than test takers as a whole to indicate they had taken a commercial test prep course, suggesting that many students already view Khan Academy’s platform as a core preparation tool for the LSAT.