| Sumario: | Most massive open online courses (MOOC) use simple schemes for aggregating peer grades, taking the mean or the median, or compute weights from information other than the instructor's opinion about the students' knowledge. To reduce the difference between the instructor and students' aggregated scores, some proposals compute specific weights to aggregate the peer grades. In this work, we analyse the use of students' engagement and performance measures to compute personalized weights and study the validity of the aggregated scores produced by these common functions, mean, and median, together with two others from the information retrieval field, the geometric and harmonic means. To test this procedure, we have analysed data from a MOOC about Philosophy. The course had 1,059 students registered, and 91 participated in a peer review process that consisted in writing an essay and rating three of their peers using a rubric. We compared the aggregation scores obtained using weighted and nonweighted versions of the functions. Our results show that the correlation between the aggregated scores and the instructor's grades can be improved in relation to peer grading, when using the median and weights are computed according to students' performance in chapter tests. Lay Description: What is already known about this topic: Several MOOC platforms introduce peer review to grade their large number of students enrolled.Peer review makes learners evaluate and grade the work of their equals.MOOCs use unweighted aggregation schemes, for example, mean or median, or compute weights from information other than the instructor's opinion about the students' knowledge. What this paper adds: Different weighted aggregation schemes are applied and analysed to reduce the difference with the grades of a potential instructor.We compute weights proportional to students' engagement or learners' marks on questionnaires.Our approach is also compared with a recent approach for aggregating peer scores. Implications for practice and/or policy: Our approach produced marks closer to those of the instructor with regard to the unweighted aggregations and a recent approach.Contrary to other approaches, our proposal does not increase students' workload.
|
|---|