A Hierarchical Rater Model Approach for Integrating Automated Essay Scoring Models.
Abstract: Essay writing tests, integral in many educational settings, demand significant resources for manual scoring. Automated essay scoring (AES) can alleviate this by automating the process, thereby reducing human effort. However, the multitude of AES models, each varying in its features and sco...
| Publicado en: | Zeitschrift für Psychologie Vol. 232; no. 3; pp. 209 - 219 |
|---|---|
| Autores principales: | , , , , |
| Formato: | equations & formulas research tables/charts Journal Article |
| Publicado: |
Hogrefe Publishing GmbH
2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=178417823&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 178417823 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 21908370 ETX8 jtl: Zeitschrift für Psychologie issn: 21908370 maglogo: N pubinfo: dt: 2024 vid: 232 iid: 3 pid: 56293 pub: Hogrefe Publishing GmbH place: Göttingen, <Blank> artinfo: ui: 178417823 178417823 178417823 10.1027/2151-2604/a000567 178417823 ppf: 209 ppct: 10 formats: tig: atl: A Hierarchical Rater Model Approach for Integrating Automated Essay Scoring Models. aug: au: Fink, Aron Gombert, Sebastian Liu, Tuo Drachsler, Hendrik Frey, Andreas affil: Educational Psychology: Counseling, Measurement, & Evaluation, Institute of Psychology, Goethe University Frankfurt, Frankfurt a. M., Germany sug: subj: Educational Measurement Computerized Educational Testing Writing Task Performance and Analysis Human Female Male Models, Educational Natural Language Processing Item Analysis Measurement Issues and Assessments Correlation Coefficient Descriptive Statistics Funding Source Female Male ab: Abstract: Essay writing tests, integral in many educational settings, demand significant resources for manual scoring. Automated essay scoring (AES) can alleviate this by automating the process, thereby reducing human effort. However, the multitude of AES models, each varying in its features and scoring approaches, complicates selecting one optimal model, especially when evaluating diverse content-related aspects across multiple rating items. Therefore, we propose a hierarchical rater model-based approach to integrate predictions from multiple AES models, accounting for their distinct scoring behaviors. We investigated its performance on data from a university essay writing test. The proposed method achieved accuracy that was comparable to the best individual AES model. This is a promising result because it additionally reduced the amount of differential item functioning between human and automated scoring and thus established a higher degree of measurement invariance compared to the individual AES models. pubtype: Academic Journal doctype: equations & formulas research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|