Research Reports

Applications of Uniform Test Assembly for the LSAT (RR 05-01)

In a large-scaled high-stakes testing program such as the Law School Admission Test (LSAT), it is necessary to maintain a large bank of test items to support the demand for a new test form at nearly every administration. To assure that the item bank can support the test assembly requirements, ongoing monitoring of the quality of the item bank is necessary to identify deficiencies and direct item development efforts. Recent research along these lines has included efforts to identify the properties of the most valuable item in the item pool, identify the test assembly constraint(s) that are the most difficult to meet, determine the distribution of test taker ability that supports the highest degree of usability of the item pool, and develop statistical test assembly targets for multiple stage testing.

Many of these practical testing problems have recently been addressed by the application of test sampling methods. Test sampling may be described as the sequential assembly of multiple test forms such that each question (item) or item set can be used an unlimited number of times. Therefore, test forms produced can overlap with each other, i.e., have items/item sets in common. The result is a sample of test forms from the finite set of all test forms available from the given item pool under a given set of test assembly constraints. In order to insure that the inferences from such research are statistically correct, the sampling must be uniform, that is, each test should have an equal chance of being assembled. Thus, the test assembly problem plays a fundamental role in the test sampling method. In particular, our interest is in methods of uniform test assembly providing uniform test sampling.

Mixed integer programming is an approach commonly applied to the test assembly problem. This paper presents proof that the mixed integer programming approach cannot guarantee uniformity of the sampling, and goes on to formulate a test assembly algorithm that assures a uniform sampling. An extension for assembling multiple nonoverlapping test forms/test sections based on item usage frequency is described, as are extensions to multiple stage and computerized adaptive testing.

The methods illustrated in the paper provide researchers and practitioners from testing organizations with a simple and flexible framework for assembling tests and monitoring item pools in linear, multiple stage, and computerized adaptive testing.

Request the full report

Additional reports in this collection

researchers study paperwork and examine charts and figures on a tablet

Evidence to Support Validity Claims for Using LSAT Scores...

Law School Admission Test (LSAT) scores provide a standard measure of an applicant’s proficiency in a well-defined set of important skills associated with success in law school coursework. LSAT scores are also a strong predictor of first-year grades (FYG) and cumulative grade point average (CGPA) in law school. The most recent correlational study of LSAT results (2019) shows that LSAT scores are far superior to undergraduate grade point average (UGPA) in predicting FYG...

Understanding and Interpreting Law School Enrollment Data...

The Law School Admission Council (LSAC) has a long-standing commitment to diversity, equity, and inclusion in legal education and in the legal profession. In line with its mission to promote quality, access, and equity in legal education, LSAC is providing this report, Understanding and Interpreting Law School Enrollment Data: A Focus on Race and Ethnicity, to help law schools, admission professionals, and other legal education stakeholders understand how we are measuring who is the pipeline.