Robustness of Adaptive Measurement of Change to Item Parameter Estimation Error.
Adaptive measurement of change (AMC) is a psychometric method for measuring intra-individual change on one or more latent traits across testing occasions. Three hypothesis tests—a Z test, likelihood ratio test, and score ratio index—have demonstrated desirable statistical properties in this context,...
| Published in: | Educational & Psychological Measurement Vol. 82; no. 4; pp. 643 - 678 |
|---|---|
| Main Authors: | , , |
| Format: | Article |
| Published: |
Sage Publications Inc.
Aug2022
|
| Subjects: | |
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=157611910&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 157611910 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Aug2022 vid: 82 iid: 4 pid: 344 pub: Sage Publications Inc. artinfo: ui: 157611910 10.1177/00131644211033902 ppf: 643 ppct: 35 formats: tig: atl: Robustness of Adaptive Measurement of Change to Item Parameter Estimation Error. aug: au: Cooperman, Allison W. Weiss, David J. Wang, Chun affil: University of Minnesota–Twin Cities, Minneapolis, MN, USA University of Washington, Seattle, WA, USA su: Psychometrics Computer adaptive testing Research methodology evaluation Mathematical models Comparative studies Theory Descriptive statistics Statistical models Data analytics Measurement errors sug: subj: Psychometrics Computer adaptive testing Research methodology evaluation Mathematical models Comparative studies Theory Descriptive statistics Statistical models Data analytics Measurement errors keyword: adaptive measurement of change computerized adaptive testing item parameter estimation error adaptive measurement of change computerized adaptive testing item parameter estimation error ab: Adaptive measurement of change (AMC) is a psychometric method for measuring intra-individual change on one or more latent traits across testing occasions. Three hypothesis tests—a Z test, likelihood ratio test, and score ratio index—have demonstrated desirable statistical properties in this context, including low false positive rates and high true positive rates. However, the extant AMC research has assumed that the item parameter values in the simulated item banks were devoid of estimation error. This assumption is unrealistic for applied testing settings, where item parameters are estimated from a calibration sample before test administration. Using Monte Carlo simulation, this study evaluated the robustness of the common AMC hypothesis tests to the presence of item parameter estimation error when measuring omnibus change across four testing occasions. Results indicated that item parameter estimation error had at most a small effect on false positive rates and latent trait change recovery, and these effects were largely explained by the computerized adaptive testing item bank information functions. Differences in AMC performance as a function of item parameter estimation error and choice of hypothesis test were generally limited to simulees with particularly low or high latent trait values, where the item bank provided relatively lower information. These simulations highlight how AMC can accurately measure intra-individual change in the presence of item parameter estimation error when paired with an informative item bank. Limitations and future directions for AMC research are discussed. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|