| Sumario: | Background: While prior research has shown that timely and personalised feedback improves students' learning outcomes and self‐regulation, most existing systems fail to provide actionable, individualised explanations at scale, especially in programming education. Manual feedback is resource‐intensive, and traditional Artificial Intelligence (AI) systems often lack transparency, limiting their pedagogical value. Objectives: This study addresses these gaps by leveraging Large Language Models (LLMs) and Explainable AI (XAI)—specifically, the SHapley Additive exPlanations (SHAP) method—to generate interpretable, scalable feedback that enhances self‐regulated learning (SRL) in the context of programming education. Methods: In the present study, behavioural data were collected from BookRoll, an e‐reading system that tracks interactions like highlighting and note‐taking, and VisCode, a coding platform that records compile attempts, error types and code execution behaviour. Combined with self‐reported strategy data, these formed the LBLS dataset used to train a predictive model. SHAP was used to identify key learning features, which were then input into the GPT‐4 model to generate personalised mid‐semester reports. Results: Results showed that students receiving LLM‐generated suggestions improved in SRL behaviours and final performance. Most found the feedback understandable and useful, though some questioned its accuracy. Conclusion: This study demonstrates the potential of combining LLMs and XAI to deliver meaningful, scalable feedback, but also highlights the need for human oversight. Practitioner Notes: What is currently known about this topic? ○Self‐regulated learning (SRL) is crucial for student success in programming education.○Timely and personalised feedback is key to supporting SRL, but traditional manual feedback is difficult to scale.○While AI can be used for feedback, its 'black box' nature often makes it difficult to interpret, reducing its educational value.○Explainable AI (XAI) can increase the transparency of AI predictions, but its outputs can be too complex for non‐technical learners.What does this paper add? ○This study presents an innovative approach that combines the analytical power of XAI (SHAP) with the natural language generation capabilities of a Large Language Model (GPT‐4) to provide scalable and comprehensible personalised feedback in programming education.○The research confirms that this combined XAI‐LLM method can effectively translate complex learning behaviour data into specific, actionable suggestions that students can understand and adopt.○Findings indicate that students who received LLM‐generated feedback showed significant improvements in self‐regulated learning strategies (such as help‐seeking, rehearsal and metacognitive monitoring) and academic performance in programming.Implications of study findings for practitioners ○Educators and instructional designers can consider adopting systems that integrate XAI and LLMs to provide students with large‐scale, automated and personalised learning support.○In programming courses, providing feedback that explains how student learning behaviours impact their performance can effectively promote metacognition and self‐regulation.○While AI‐generated feedback has great potential, educators still play a critical role in the system by supervising the quality and accuracy of the feedback and providing additional guidance when necessary.○When developing or selecting AI educational tools, priority should be given to systems that not only provide predictions but also explain the reasoning behind them in a learner‐centric manner.
|
|---|