| Sumario: | Background: This research employs the concept of digital maturity (DM) to characterise levels of digital use in education and to represent them on a dashboard. This dashboard should enable assessment of teachers' digital practices and tracking of their technology adoption over time, particularly when new continuing professional development opportunities are offered to support their integration of digital tools into their professional practice. The challenge of this research is to make these observations at scale and in a regular manner. This study explores the use of Teaching Analytics to measure and report on teachers' DM. Objectives: The paper proposes methods for processing and analysing teachers' virtual learning environment (VLE) activity logs to characterise DM. Specifically, the focus is on identifying analytical methods for modelling and assessing maturity, as well as visualising population states to help stakeholders better understand and use these data. Methods: The techniques employed include structuring data according to the DigCompEdu competence framework, supervised and unsupervised classification to assess maturity levels across dimensions (diversity, intensity, and type of usage), and three visualisations (Sunburst, Histogram, and Bubble chart) integrated into a dashboard. These three visualisations were evaluated (usability and satisfaction, perceived credibility, perceived usefulness, and actionability) during interviews with stakeholders. A discussion also addresses interactive methods for presenting these results through a dashboard. The methods are illustrated using increasingly large datasets representing 12,949 teachers (Paris, 2022–2023), 144,900 teachers (Paris, Lille, Amiens, 2022–2024), and 1,320,000 teachers (national area, 2022–2026). Results and Conclusions: This study makes three contributions. First, it proposes an operational framework for designing a multidimensional dashboard that uses VLE logs to assess teachers' DM at scale. It uses supervised and unsupervised classification and considers the intensity of use (frequency) of the VLE platform by teachers, as well as the diversity and types of use. Second, it compares supervised and unsupervised classification approaches to examine their scalability and analytical accuracy. Third, it explores how different institutional actors perceive dashboards, focusing on their interpretability, credibility, and practical usefulness. Lay Summary: What is currently known about this topic? ○The concept of DM is applied in organisational contexts and is beginning to be explored in the educational field, but many definitions and evaluation methods remain poorly established.○Large‐scale measurement of DM remains largely unexplored.○Teaching Analytics provides valuable insights for evaluating the use of educational technologies; however, there are no established analytical methods for assessing DM.○What does this paper add?○A multidimensional framework for assessing teachers' DM using large‐scale VLE activity logs.○Data modelling according to the DigCompEdu competence framework, which is recognised in Europe for characterising teachers' digital professional practices.○Two automatic classification methods (supervised and unsupervised) are proposed to identify maturity classes. The intensity (frequency) of use of digital services, diversity, and type of usage are employed to characterise maturity levels.○Interactive visualisations that support the exploration and comparison of DM on a large scale.○Qualitative insights into the perceived usefulness and relevance of dashboards, based on users' roles and contexts of use.Implications for Practice/or Policy ○Evidence‐based insights into the DM of a large population of primary and secondary school teachers, supporting informed decision‐making at institutional and territorial levels.○Design guidelines for Teaching Analytics dashboards that account for the needs and practices of diverse educational stakeholders (e.g., strategic leaders, training professionals, and support staff). Highlights: Proposes a multidimensional framework assessing teachers' DM from large‐scale VLE logs.Demonstrates scalability using datasets up to 1,320,000 teachers across France (2022–2026).Designs visualisations and deploys an operational Teaching Analytics dashboard for institutional stakeholders.Reveals low levels of teachers' DM, with practices predominantly limited to communication uses.Shows, through an empirical study, that dashboards support decision‐making but require stakeholder‐specific adaptation.
|
|---|