LSAC Resources - Research

A Study of Structural Modeling Using Plausible Value Imputation (RR 08-03)

by Cees A. W. Glas and Hanneke Geerlings, University of Twente, Enschede, The Netherlands

Executive Summary

In computerized adaptive testing, the responses of test takers to test questions (items) are used to select subsequent items for administration that are tailored to the test taker's ability level. A mathematical model called item response theory (IRT) is commonly used to estimate both the characteristics of the items and the ability level of the test takers. Data from computerized adaptive tests can be used to evaluate hypotheses about student proficiency, such as hypotheses about subgroup differences (gender, racial/ethnic group, previous education, preparatory training) or the development of their proficiencies. The hypotheses can be evaluated by adding a structural model for test taker ability to the IRT model. Typical examples of such structural models are (linear) analysis of variance and regression models.

The goal of this study is to compare the power of several common but complex methods for such extended models (i.e., marginal maximum or Bayesian methods) to a simpler alternative (i.e., plausible value imputation). The power of the methods is evaluated for IRT models for dichotomously and polytomously scored items, and for a model for responses to dichotomous items combined with response times. Simulation studies show that a relatively simple version of the plausible value imputation method does not perform worse than more advanced methods.

• Full Research Report (PDF)