On Bank Assembly and Block Selection in Multidimensional Forced-Choice Adaptive Assessments.
Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theo...
| Publicado en: | Educational & Psychological Measurement Vol. 83; no. 2; pp. 294 - 322 |
|---|---|
| Autores principales: | , , |
| Formato: | Artículo |
| Publicado: |
Sage Publications Inc.
Apr2023
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=162054941&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 162054941 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Apr2023 vid: 83 iid: 2 pid: 344 pub: Sage Publications Inc. artinfo: ui: 162054941 10.1177/00131644221087986 ppf: 294 ppct: 28 formats: tig: atl: On Bank Assembly and Block Selection in Multidimensional Forced-Choice Adaptive Assessments. aug: au: Kreitchmann, Rodrigo S. Sorrel, Miguel A. Abad, Francisco J. affil: Universidad Autónoma de Madrid, Spain su: Personality disorder diagnosis Analysis of variance Psychological adaptation Computer adaptive testing Research evaluation Research methodology Research methodology evaluation Simulation methods in education Questionnaires Descriptive statistics Research funding Prediction models Data analysis software Algorithms Evaluation sug: subj: Personality disorder diagnosis Analysis of variance Psychological adaptation Computer adaptive testing Research evaluation Research methodology Research methodology evaluation Simulation methods in education Questionnaires Descriptive statistics Research funding Prediction models Data analysis software Algorithms Evaluation keyword: adaptive testing forced-choice format ipsative data item selection multidimensional IRT adaptive testing forced-choice format ipsative data item selection multidimensional IRT ab: Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled on-the-fly considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A -rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A -rule with questionnaires assembled on-the-fly, the T -rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|