Study Shows LSAT Score Increases for Candidates Who Use Free Khan Academy Prep Tools
The more time law school candidates spent preparing for the Law School Admission Test via Khan Academy’s free online LSAT preparation tools, and the more practice exams those candidates completed on the Khan Academy platform, the higher the candidates tended to score on the LSAT. This is according to a recently completed Law School Admission Council study that appears to reinforce the value of this online resource for candidates.
While the study could not rule out alternative influences on LSAT performance, LSAC researchers Kimberly Dustman and Gregory Camilli wrote that “engagement with the Khan Academy platform is currently the best explanation for the LSAT score increases observed in this study.”
In looking at the time period from June 2018 (when Khan Academy’s LSAT prep tool was launched) to July 2020, Dustman and Camilli matched Khan Academy usage information with data from LSAT test takers who indicated they used Khan Academy to prepare for the test and gave LSAC permission to use their data for research purposes. This yielded a sample of 12,471 candidates whose experiences with Khan Academy and on the LSAT were compared with those of all test takers from the same time period.
The researchers focused on two variables: the minutes candidates spent on Khan Academy practice activities, and the number of practice exams candidates took. They found a moderate, but positive and statistically significant, correlation between practice activity minutes and LSAT scores; in other words, the longer a candidate spent completing Khan Academy practice activities, the higher they tended to score on the LSAT.
Similarly, taking more practice LSAT exams, up to the maximum of 10 allowed on the platform, “generally resulted in increased LSAT scores relative to number of practice exams taken,” the researchers wrote. The study also found it unlikely that other influences, such as commercial LSAT prep courses, could have accounted for those increased scores.
Importantly, Dustman and Camilli found that the positive effects of using the Khan Academy prep tools were present across demographic subgroups. In fact, the researchers noted that Black, Hispanic, and female candidates had the most pronounced association between number of practice exams and LSAT score, suggesting that the practice exams are of particular value for these groups that have been historically underrepresented in the legal profession.
For example, the study found that Black law school candidates who completed one or two practice tests saw an increase of 2.84 points, on average, in their LSAT score, while Black candidates who finished nine or ten practice LSATs saw an average increase of a full 8 points. Hispanic and female candidates saw similar results, while increases for White, Asian, and male candidates were present but slightly less pronounced.
In the overall study sample, candidates who completed one or two practice LSATs saw an average score increase of 1.59 points on the actual LSAT. That score boost increased to 3.58 points with three or four practice tests, 4.39 points with five or six practice tests, 5.59 points with seven or eight practice tests, and 7.26 points with nine or ten practice tests.
Roughly half of the students in the study sample did not complete a full Khan Academy practice exam, and Dustman and Camilli stated that further research is needed to understand how to better engage students on the platform — and to further study the effectiveness of that platform. Additionally, the researchers cautioned that Khan Academy practice activities could not yet be declared the cause of LSAT score increases; the study, which was described as “quasi-experimental,” simply shows a significant correlation between those two variables.
The Khan Academy platform allows candidates to create personalized practice plans that cater to each candidate's strengths and weaknesses. It includes interactive lessons, problem sets, timed practice exams, and videos. In September of 2020, nearly 70,000 people used the Khan Academy platform in some capacity.
“Because the Khan LSAT platform is free of charge and easily accessible online, individuals preparing to take the LSAT may want to consider incorporating Khan Academy’s Official LSAT Prep activities,” the researchers wrote, adding that the Khan Academy platform “is likely a pragmatic alternative to commercial test prep courses and private tutoring.” In fact, the sample of Khan users was less likely than test takers as a whole to indicate they had taken a commercial test prep course, suggesting that many students already view Khan Academy’s platform as a core preparation tool for the LSAT.