How can valid and reliable automatic formative assessment predict the acquisition of learning outcomes?
Background: Sound learning design should be based on the constructive alignment of intended learning outcomes (LOs), teaching and learning activities and formative and summative assessment. Assessment validity strongly relies on its alignment with LOs. Valid and reliable formative assessment can be...
| Publicado en: | Journal of Computer Assisted Learning Vol. 40; no. 6; pp. 2616 - 2633 |
|---|---|
| Autores principales: | , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Dec2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=180899658&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 180899658 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Dec2024 vid: 40 iid: 6 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 180899658 175495159 180899658 180899658 10.1111/jcal.12953 180899658 ppf: 2616 ppct: 17 formats: tig: atl: How can valid and reliable automatic formative assessment predict the acquisition of learning outcomes? aug: au: Divjak, Blaženka Svetec, Barbi Horvat, Damir affil: Faculty of Organization and Informatics, University of Zagreb, Varaždin, Croatia sug: subj: Outcomes of Education Educational Measurement Formative Evaluation Research Automation Student Knowledge Evaluation Human Prediction Models Reliability and Validity Random Forest Mathematics Data Quality Factor Analysis Learning Environment Data Collection ROC Curve Models, Educational Textbooks Motion Presenteeism Funding Source ab: Background: Sound learning design should be based on the constructive alignment of intended learning outcomes (LOs), teaching and learning activities and formative and summative assessment. Assessment validity strongly relies on its alignment with LOs. Valid and reliable formative assessment can be analysed as a predictor of students' academic performance, but the question is how significant its predictive power is, and what other elements can affect predictions. Objectives: Our aim was to investigate the predictive power of formative assessment for summative assessment, measuring the acquisition of LOs. Methods: We analysed formative assessment results (quizzes, homework), together with log data (video and other material use, class attendance), to determine the most influential predictors and establish a reliable predictive learning analytics model. We used the Random Forest algorithm. The model is based on the data from two university mathematical courses, delivered at different years and levels of study, incorporating 813 students in two consecutive years. Results and Conclusions: Our results show that formative assessment, together with previous summative assessment, is a stronger predictor of summative assessment results than other data on students' engagement. The study pointed to the importance of completeness and quality of data, and clear links between assessment and LOs when making predictions of student results. It suggested that predictions are less reliable for the lowest and the highest performing students. It was noted that other factors can also affect predictions, like the level of LOs, or factors not easily extracted from digital data, like the learning environment and individual students' strategies. Lay Description: What is currently known about this topic?: When educators design lessons, they aim for students to acquire intended learning outcomes.To know if students are making progress, educators use a mix of regular checks, like quizzes (formative assessment) and major exams (summative assessment).Predictive learning analytics may contribute to sound learning design (LD). What does this paper add?: In our study, we looked into whether these regular checks can predict how a student might perform on high‐stake exams.We collected data from students taking two university maths courses, including their scores on quizzes, their attendance, and their use of study materials.Using a machine learning technique called the Random Forest algorithm, we built a model to predict student exam scores.Our results indicate that formative assessment is the best predictor of the success in summative assessment. Implications for practice/or policy: We found that how a student performs on regular checks is a good sign of how they might do on the high‐stake exam and careful preparation of assessment and whole LD help student learning.Assessment, formative as well as summative, needs to be aligned with intended learning outcomes and only in this case results can inform quality assurance processes and the restructuring of LD.There are other personal factors that can influence a student's performance, which are not easy to spot just from the digital data we have.The complexity of what is being taught plays a role in predicting students' final scores. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|