Research Reports

Comparison Study of Item Preknowledge Detectors (RR 14-01)

When a test taker has prior knowledge about an administered test question (item), then this event is called item preknowledge, the test taker is called aberrant, and the item is called compromised. Item preknowledge negatively affects the corresponding testing program and its test score users (universities, companies, government organizations) because the scores produced for aberrant test takers will be invalid. The performance of eight statistics for detection of item preknowledge (five existing, two modified, and one new) was studied via computer simulations. Three major factors that could potentially influence the performance of the statistics were considered: (a) the type of test (adaptive, in which the next administered item is selected based on the test taker’s responses to previously administered items; or nonadaptive, in which all test takers are administered the same group of items); (b) distribution of the aberrant population (normal or uniform); and (c) noise in the information about compromised items (since different groups of aberrant test takers may have prior knowledge of different groups of items). The last factor demonstrated the highest negative impact on the performance of all of the statistics: the greater the noise, the lower the detection of item preknowledge. Several methods to address this problem are discussed.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.