The role of rapid guessing and test‐taking persistence in modelling test‐taking engagement.
Background: Item response times in computerized assessments are frequently used to identify rapid guessing behaviour as a manifestation of response disengagement. However, non‐rapid responses (i.e., with longer response times) are not necessarily engaged, which means that response‐time‐based procedu...
| Publicado en: | Journal of Computer Assisted Learning Vol. 39; no. 3; pp. 751 - 767 |
|---|---|
| Autores principales: | , , |
| Formato: | equations & formulas research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Jun2023
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=163886536&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 163886536 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Jun2023 vid: 39 iid: 3 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 163886536 158580005 163886536 163886536 10.1111/jcal.12719 163886536 ppf: 751 ppct: 16 formats: tig: atl: The role of rapid guessing and test‐taking persistence in modelling test‐taking engagement. aug: au: Nagy, Gabriel Ulitzsch, Esther Lindner, Marlit Annalena affil: Educational Measurement, Leibniz Institute for Science and Mathematics Education, Kiel, Germany sug: subj: Computerized Educational Testing Behavior Evaluation Students, Elementary Psychosocial Factors Decision Making Student Attitudes Human Reaction Time Theory Science Academic Performance Self Report Models, Theoretical Probability Random Assignment Regression Child Male Female Teaching Materials Pamphlets Descriptive Statistics Software Maximum Likelihood Algorithms Funding Source Child: 6-12 years Male Female ab: Background: Item response times in computerized assessments are frequently used to identify rapid guessing behaviour as a manifestation of response disengagement. However, non‐rapid responses (i.e., with longer response times) are not necessarily engaged, which means that response‐time‐based procedures could overlook disengaged responses. Therefore, the identification of disengaged responses could be improved by considering additional indicators of disengagement. We investigated the extent to which decreases in individuals' item solution probabilities over the course of a test reflect disengaged response behaviour. Objectives: To disentangle different types of possibly disengaged responses and better understand non‐effortful test‐taking behaviour, we augmented responses‐time‐based procedures for identifying rapid guessing with strategies for detecting disengaged responses on the basis of performance declines in non‐rapid responses. Methods: We combined item response theory (IRT) models for rapid guessing and test‐taking persistence to examine the capability of response times and item positions to capture response disengagement. We used a computerized assessment in which science items were randomly distributed across positions for each student. This allowed us to estimate individual differences in test‐taking persistence (i.e., the duration for which the initial level of performance is maintained) while accounting for rapid responses. Results and Conclusions: Response times did not fully explain disengagement; item responses reflected test‐taking persistence even when rapid responses were accounted for. This interpretation was supported by a strong correlation of test‐taking persistence with decreases in self‐reported test‐taking effort. Furthermore, our results suggest that IRT models for test‐taking persistence can effectively account for the undesirable impact of low test‐taking effort even when response times are unavailable. Practitioner Notes: Assessments of proficiencies that attempt to quantify what individuals know and can do lead to biased results when individuals provide disengaged responses.Item response times are frequently used to identify rapid guessing behaviour as a manifestation of response disengagement, but response‐time‐based procedures could overlook disengaged responses.To disentangle different types of possibly disengaged responses and better understand non‐effortful test‐taking behaviour, we augmented responses‐time‐based procedures for identifying rapid guessing with strategies for detecting disengaged responses on the basis of performance declines in non‐rapid responses.In a sample of fifth and sixth graders, we found that response times did not fully explain disengagement, as many students showed performance declines in non‐rapid item responses.Our results suggest that item response theory models for test‐taking persistence can effectively account for the undesirable impact of low test‐taking effort even when response times are unavailable. pubtype: Academic Journal doctype: equations & formulas research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|