Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study.
Purpose: This study aimed to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under 2 stopping rules (standard error of measurement [SEM]=0.3 and 0.25) using both real and simulated data in medical examinations in Korea. Methods: This study employed post-hoc si...
| Publicado en: | Journal of Educational Evaluation for Health Professions Vol. 21; pp. 1 - 7 |
|---|---|
| Autores principales: | , , |
| Formato: | equations & formulas pictorial research tables/charts Journal Article |
| Publicado: |
National Health Personnel Licensing Examination Board
2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=182175674&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 182175674 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 19755937 B0CC jtl: Journal of Educational Evaluation for Health Professions issn: 19755937 maglogo: N pubinfo: dt: 2024 vid: 21 pid: 58691 pub: National Health Personnel Licensing Examination Board artinfo: ui: 182175674 182175674 182175674 10.3352/jeehp.2024.21.18 182175674 ppf: 1 ppct: 6 formats: fmt: @attributes: type: P tig: atl: Comparison of real data and simulated data analysis of a stopping rule based on the standard error of measurement in computerized adaptive testing for medical examinations in Korea: a psychometric study. aug: au: Dong Gi Seo Jeongwook Choi Jinha Kim affil: Department of Psychology, Hallym Applied Psychology Institute, College of Social Science, Hallym University, Chuncheon, Korea sug: subj: Computerized Adaptive Testing Measurement Error Standards Psychometrics Evaluation Clinical Assessment Tools Human Funding Source South Korea Comparative Studies Post Hoc Analysis Simulations Students, Medical Academic Medical Centers Descriptive Statistics Models, Statistical Educational Measurement ab: Purpose: This study aimed to compare and evaluate the efficiency and accuracy of computerized adaptive testing (CAT) under 2 stopping rules (standard error of measurement [SEM]=0.3 and 0.25) using both real and simulated data in medical examinations in Korea. Methods: This study employed post-hoc simulation and real data analysis to explore the optimal stopping rule for CAT in medical examinations. The real data were obtained from the responses of 3rd-year medical students during examinations in 2020 at Hallym University College of Medicine. Simulated data were generated using estimated parameters from a real item bank in R. Outcome variables included the number of examinees' passing or failing with SEM values of 0.25 and 0.30, the number of items administered, and the correlation. The consistency of real CAT result was evaluated by examining consistency of pass or fail based on a cut score of 0.0. The efficiency of all CAT designs was assessed by comparing the average number of items administered under both stopping rules. Results: Both SEM 0.25 and SEM 0.30 provided a good balance between accuracy and efficiency in CAT. The real data showed minimal differences in pass/fail outcomes between the 2 SEM conditions, with a high correlation (r=0.99) between ability estimates. The simulation results confirmed these findings, indicating similar average item numbers between real and simulated data. Conclusion: The findings suggest that both SEM 0.25 and 0.30 are effective termination criteria in the context of the Rasch model, balancing accuracy and efficiency in CAT. pubtype: Academic Journal doctype: equations & formulas pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|