The impact of differential item functioning on ability estimation using the Korean Medical Licensing Examination with computerized adaptive testing: a post-hoc simulation study.
Purpose: This study examined the impact of differential item functioning (DIF) on ability estimation in a computerized adaptive testing (CAT) environment using real response data from the 2017 Korean Medical Licensing Examination (KMLE). We hypothesized that excluding gender-based DIF items would im...
| Publicado en: | Journal of Educational Evaluation for Health Professions Vol. 22; pp. 1 - 9 |
|---|---|
| Autores principales: | , , |
| Formato: | equations & formulas pictorial research tables/charts Journal Article |
| Publicado: |
National Health Personnel Licensing Examination Board
2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=192170966&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 192170966 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 19755937 B0CC jtl: Journal of Educational Evaluation for Health Professions issn: 19755937 maglogo: N pubinfo: dt: 2025 vid: 22 pid: 58691 pub: National Health Personnel Licensing Examination Board artinfo: ui: 192170966 192170966 192170966 10.3352/jeehp.2025.22.31 192170966 ppf: 1 ppct: 8 formats: fmt: @attributes: type: P tig: atl: The impact of differential item functioning on ability estimation using the Korean Medical Licensing Examination with computerized adaptive testing: a post-hoc simulation study. aug: au: Kim, Dogyeong Choi, Jeongwook Seo, Dong Gi affil: Department of Psychology, College of Social Science, Hallym University, Chuncheon, Korea sug: subj: Education, Medical South Korea Credentialing Examinations Licensure Educational Measurement Computerized Educational Testing Differential Item Functioning Psychometrics Evaluation Human Funding Source South Korea Male Female Post Hoc Analysis Simulations Sex Factors Discrimination Validity Evaluation Descriptive Statistics Pearson's Correlation Coefficient Rasch Analysis Data Analysis Software Male Female ab: Purpose: This study examined the impact of differential item functioning (DIF) on ability estimation in a computerized adaptive testing (CAT) environment using real response data from the 2017 Korean Medical Licensing Examination (KMLE). We hypothesized that excluding gender-based DIF items would improve estimation accuracy, particularly for examinees at the extremes of the ability scale. Methods: The study was conducted in 2 steps: (1) DIF detection and (2) post-hoc simulation. The analysis used data from 3,259 examinees who completed all 360 dichotomous items. Gender-based DIF was detected with the residual-based DIF method (reference group: males; focal group: females). Two CAT conditions (all items vs. DIF-excluded) were compared against a "true θ" estimated from a fixed-form test of 264 non-DIF items. Accuracy was evaluated using bias, root mean square error (RMSE), and correlation with true θ. Results: In the CAT condition excluding DIF items, accuracy improved, with RMSE reduced and correlation with true θ increased. However, bias was slightly larger in magnitude. Gender-specific analyses showed that DIF removal reduced the underestimation of female ability but increased the underestimation of male ability, yielding estimates that were fairer across genders. When DIF items were included, estimation errors were more pronounced at both low and high ability levels. Conclusion: Managing DIF in CAT-based high-stakes examinations can enhance fairness and precision. Using real examinee data, this study provides practical evidence of the implications of DIF for CAT-based measurement and supports fairness-oriented test design. pubtype: Academic Journal doctype: equations & formulas pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|