You're (Not) My Type‐ Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?
Background: Feedback as one of the most influential factors for learning has been subject to a great body of research. It plays a key role in the development of educational technology systems and is traditionally rooted in deterministic feedback defined by experts and their experience. However, with...
| Publicado en: | Journal of Computer Assisted Learning Vol. 41; no. 1; pp. 1 - 21 |
|---|---|
| Autores principales: | , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Feb2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=183981456&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 183981456 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Feb2025 vid: 41 iid: 1 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 183981456 183981456 183981456 10.1111/jcal.13107 183981456 ppf: 1 ppct: 20 formats: tig: atl: You're (Not) My Type‐ Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks? aug: au: Lohr, Dominic Keuning, Hieke Kiesler, Natalie affil: Friedrich‐Alexander‐Universität Erlangen, Nürnberg, Germany sug: subj: Artificial Intelligence, Generative Programming Languages Education Feedback Classification Uncertainty Educational Technology Human Comparative Studies Qualitative Studies Summated Rating Scaling Students Psychosocial Factors Learning Methods Self Regulation Java Problem Solving Readability Semantics ab: Background: Feedback as one of the most influential factors for learning has been subject to a great body of research. It plays a key role in the development of educational technology systems and is traditionally rooted in deterministic feedback defined by experts and their experience. However, with the rise of generative AI and especially large language models (LLMs), we expect feedback as part of learning systems to transform, especially for the context of programming. In the past, it was challenging to automate feedback for learners of programming. LLMs may create new possibilities to provide richer, and more individual feedback than ever before. Objectives: This article aims to generate specific types of feedback for introductory programming tasks using LLMs. We revisit existing feedback taxonomies to capture the specifics of the generated feedback, such as randomness, uncertainty and degrees of variation. Methods: We iteratively designed prompts for the generation of specific feedback types (as part of existing feedback taxonomies) in response to authentic student programs. We then evaluated the generated output and determined to what extent it reflected certain feedback types. Results and Conclusion: This study provides a better understanding of different feedback dimensions and characteristics. The results have implications for future feedback research with regard to, for example, feedback effects and learners' informational needs. It further provides a basis for the development of new tools and learning systems for novice programmers including feedback generated by AI. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|