Harnessing the potential of trace data and linguistic analysis to predict learner performance in a multi‐text writing task.
Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while de...
| Publicado en: | Journal of Computer Assisted Learning Vol. 39; no. 3; pp. 703 - 719 |
|---|---|
| Autores principales: | , , , , , , , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Jun2023
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=163886539&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 163886539 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Jun2023 vid: 39 iid: 3 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 163886539 160751645 163886539 163886539 10.1111/jcal.12769 163886539 ppf: 703 ppct: 16 formats: tig: atl: Harnessing the potential of trace data and linguistic analysis to predict learner performance in a multi‐text writing task. aug: au: Raković, Mladen Iqbal, Sehrish Li, Tongguang Fan, Yizhou Singh, Shaveen Surendrannair, Surya Kilgour, Jonathan van der Graaf, Joep Lim, Lyn Molenaar, Inge Bannert, Maria Moore, Johanna Gašević, Dragan affil: Centre for Learning Analytics, Monash University, Melbourne Victoria,, Australia sug: subj: Machine Learning Academic Performance Evaluation Writing Evaluation Linguistics Evaluation Task Performance and Analysis Human Field Studies Writing Education Students, Graduate Colleges and Universities China China Adult Learning Environment Descriptive Statistics Student Assignments Algorithms Funding Source Natural Language Processing Automation Adult: 19-44 years ab: Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low‐ and high‐performing essays in a multi‐text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi‐text writing assignment. We utilised learners' trace data to obtain process features and state‐of‐the‐art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re‐reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi‐text writing task to help them do well in the task. Lay Description: What is already known about this topic?: Both product and process features should be used to inform formative feedback on writing.Providing product‐ and process‐oriented feedback to learners is challenging.Automatic writing evaluation systems have mainly relied upon product features.Automated analysis of learners' trace data and their essay drafts is a promising venue. What this paper adds?: An accurate machine learning classifier that identifies low‐ and high‐scoring essays.The classifier utilized both product and process features.We obtained process features from learners' trace data in digital learning environment.We computed product features using state‐of‐the‐art text analytical methods. Implications for practice and/or policy: The classifier can be used as a part of a future automated writing evaluation system.We revealed learning processes and essay characteristics that influence performance.Based on important predictors of performance, formative feedback can be given to learners. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|