Hybrid Threshold-Based Sequential Procedures for Detecting Compromised Items in a Computerized Adaptive Testing Licensure Exam.
Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to impr...
| Publicado en: | Educational & Psychological Measurement Vol. 82; no. 4; pp. 782 - 811 |
|---|---|
| Autores principales: | , |
| Formato: | Artículo |
| Publicado: |
Sage Publications Inc.
Aug2022
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=157611905&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 157611905 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Aug2022 vid: 82 iid: 4 pid: 344 pub: Sage Publications Inc. artinfo: ui: 157611905 10.1177/00131644211023868 ppf: 782 ppct: 29 formats: tig: atl: Hybrid Threshold-Based Sequential Procedures for Detecting Compromised Items in a Computerized Adaptive Testing Licensure Exam. aug: au: Lee, Chansoon Qian, Hong affil: Liberty University, Lynchburg, VA, USA National Council of State Boards of Nursing, Chicago, IL, USA su: Computer simulation Analysis of variance Rating of students Decision making Computer adaptive testing Professional licenses Regression analysis Conceptual structures Data security Descriptive statistics Statistical models Professional licensure examinations Probability theory sug: subj: Computer simulation Analysis of variance Rating of students Decision making Computer systems design and related services (except video game design and development) Other Computer Related Services Computer adaptive testing Professional licenses Regression analysis Conceptual structures Data security Descriptive statistics Statistical models Professional licensure examinations Probability theory keyword: change-point problem compromised item detection computerized adaptive testing test security change-point problem compromised item detection computerized adaptive testing test security ab: Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to improve the detection power of the sequential procedure while controlling the Type I error rate. The hybrid threshold approach uses a local threshold for each item in an early stage of the CAT administration, and then it uses the global threshold in the decision-making stage. Applying various simulation factors, a series of simulation studies examined which factors contribute significantly to the power rate and lag time of the procedure. In addition to the simulation study, a case study investigated whether the procedures are applicable to the real item pool administered in CAT and can identify potentially compromised items in the pool. This research found that the increment of probability of a correct answer (p -increment) was the simulation factor most important to the sequential procedures' ability to detect compromised items. This study also found that the local threshold approach improved power rates and shortened lag times when the p -increment was small. The findings of this study could help practitioners implement the sequential procedures using the hybrid threshold approach in real-time CAT administration. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|