ChatGPT in Education: An Effect in Search of a Cause.
Background: As researchers rush to investigate the potential of AI tools like ChatGPT to enhance learning, well‐documented pitfalls threaten the validity of this emerging research. Issues of media comparison research, where the confounding of instructional methods and technological affordances is un...
| Publicado en: | Journal of Computer Assisted Learning Vol. 41; no. 5; pp. 1 - 11 |
|---|---|
| Autores principales: | , , , |
| Formato: | questions and answers tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Oct2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=188234203&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 188234203 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Oct2025 vid: 41 iid: 5 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 188234203 188234203 188234203 10.1111/jcal.70105 188234203 ppf: 1 ppct: 10 formats: tig: atl: ChatGPT in Education: An Effect in Search of a Cause. aug: au: Weidlich, J. Gašević, D. Drachsler, H. Kirschner, P. affil: University of Zurich, Zurich, Switzerland sug: subj: Artificial Intelligence, Generative Teaching Methods Learning Methods Computer-Assisted Instruction Educational Technology Communications Media Problem-Based Learning Outcomes of Education Student Satisfaction ab: Background: As researchers rush to investigate the potential of AI tools like ChatGPT to enhance learning, well‐documented pitfalls threaten the validity of this emerging research. Issues of media comparison research, where the confounding of instructional methods and technological affordances is unrecognised, may render effects uninterpretable. Objectives: Using a recent meta‐analysis by Deng et al. (Computers & Education, 227, 105224) as an example, we revisit key insights from the media/methods debate to highlight recurring conceptual challenges in ChatGPT efficacy studies. Methods: This conceptual article contrasts nascent ChatGPT research with the more established literature on Intelligent Tutoring Systems to identify three non‐negotiable considerations for interpretable effects: (1) descriptions of the precise nature of the experimental treatment and (2) the activities of the control group, as well as (3) outcome measures as valid indicators of learning. To provide some initial evidence, we audited a subset of primary experiments included in Deng et al.'s meta‐analysis, demonstrating that only a small minority of studies satisfied all three non‐negotiable considerations. Results and Conclusions: Loosely defined treatments, mismatched or opaque controls, and outcome measures with unclear links to durable learning obscure causal claims of this emerging literature. Observed gains cannot, at this time, be confidently attributed to ChatGPT, and meta‐analytics effect sizes may over‐ or understate its benefits. Progress, we argue, will require rigorous designs, transparent reporting, and a critical stance toward "fast science." pubtype: Academic Journal doctype: questions and answers tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|