Exploring the Evidence to Interpret Differential Item Functioning via Response Process Data.
Evaluating differential item functioning (DIF) in assessments plays an important role in achieving measurement fairness across different subgroups, such as gender and native language. However, relying solely on the item response scores among traditional DIF techniques poses challenges for researcher...
| Published in: | Educational & Psychological Measurement Vol. 85; no. 4; pp. 783 - 814 |
|---|---|
| Main Authors: | , , , |
| Format: | Article |
| Published: |
Sage Publications Inc.
Aug2025
|
| Subjects: | |
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=186128844&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 186128844 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Aug2025 vid: 85 iid: 4 pid: 344 pub: Sage Publications Inc. artinfo: ui: 186128844 10.1177/00131644241298975 ppf: 783 ppct: 31 formats: tig: atl: Exploring the Evidence to Interpret Differential Item Functioning via Response Process Data. aug: au: Li, Ziying Shin, Jinnie Kuang, Huan Huggins-Manley, A. Corinne affil: University of Florida, Gainesville, USA Florida State University, Tallahassee, USA su: Empirical research Psychometrics Mathematical variables Random forest algorithms Differential item functioning (Research bias) Data analysis Receiver operating characteristic curves Research evaluation Logistic regression analysis Pilot projects Descriptive statistics Statistics Comparative studies Evaluation sug: subj: Empirical research Psychometrics Mathematical variables Random forest algorithms Differential item functioning (Research bias) Data analysis Receiver operating characteristic curves Research evaluation Logistic regression analysis Pilot projects Descriptive statistics Statistics Comparative studies Evaluation keyword: DIF Mantel–Haenszel random forest response process data ridge logistic regression DIF Mantel–Haenszel random forest response process data ridge logistic regression ab: Evaluating differential item functioning (DIF) in assessments plays an important role in achieving measurement fairness across different subgroups, such as gender and native language. However, relying solely on the item response scores among traditional DIF techniques poses challenges for researchers and practitioners in interpreting DIF. Recently, response process data, which carry valuable information about examinees' response behaviors, offer an opportunity to further interpret DIF items by examining differences in response processes. This study aims to investigate the potential of response process data features in improving the interpretability of DIF items, with a focus on gender DIF using data from the Programme for International Assessment of Adult Competencies (PIAAC) 2012 computer-based numeracy assessment. We applied random forest and logistic regression with ridge regularization to investigate the association between process data features and DIF items, evaluating the important features to interpret DIF. In addition, we evaluated model performance across varying percentages of DIF items to reflect practical scenarios with different percentages of DIF items. The results demonstrate that the combination of timing features and action-sequence features is informative to reveal the response process differences between groups, thereby enhancing DIF item interpretability. Overall, this study introduces a feasible procedure to leverage response process data to understand and interpret DIF items, shedding light on potential reasons for the low agreement between DIF statistics and expert reviews and revealing potential irrelevant factors to enhance measurement equity. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|