Assessing fidelity of implementation to a technology‐mediated early intervention using process data.
Background: Process data, data generated by a user's interaction with a web‐based application, is an emerging tool in educational research. The current study explores using process data as a measure of implementation fidelity to a randomized control trial (RCT) of the Read It Again Mobile (RIA‐M) cu...
| Publicado en: | Journal of Computer Assisted Learning Vol. 38; no. 2; pp. 409 - 422 |
|---|---|
| Autores principales: | , , |
| Formato: | research tables/charts randomized controlled trial Journal Article |
| Publicado: |
Wiley-Blackwell
Apr2022
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=155518135&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 155518135 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Apr2022 vid: 38 iid: 2 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 155518135 153364590 155518135 155518135 10.1111/jcal.12621 155518135 ppf: 409 ppct: 13 formats: tig: atl: Assessing fidelity of implementation to a technology‐mediated early intervention using process data. aug: au: Helsabeck, Nathan P. Justice, Laura M. Logan, Jessica A. R. affil: Crane Center for Early Childhood Education and Policy, Columbus Ohio,, USA sug: subj: Language Development Evaluation Mobile Applications Utilization Computer-Assisted Instruction Utilization Program Implementation Evaluation Teachers Psychosocial Factors Faculty Attitudes Evaluation Curriculum Development Educational Technology Early Childhood Intervention Human Female Child Randomized Controlled Trials Random Assignment Schools, Nursery Teaching Methods Pretest-Posttest Control Group Design Internet-Based Intervention Funding Source Questionnaires United States Descriptive Statistics Outcomes of Education Language Tests Child: 6-12 years Female ab: Background: Process data, data generated by a user's interaction with a web‐based application, is an emerging tool in educational research. The current study explores using process data as a measure of implementation fidelity to a randomized control trial (RCT) of the Read It Again Mobile (RIA‐M) curricular supplement. Objectives: To determine the extent to which teachers implemented RIA‐M and to assess the utility of using process data in the assessment of fidelity. Methods: The RCT involved 30 pre‐kindergarten classrooms with a sample of n = 216 students. RIA‐M provides a curricular supplement which teachers may incorporate into classroom instruction and is delivered via a tablet computer. Pre and post literacy assessments are used to determine treatment effect. Process data, produced from teacher interactions with the tablet, and classroom observations are used to assess fidelity. Results and Conclusions: Our findings indicate no difference between treatment and control students in the RCT. Yet, we find that process data provides unique fidelity information concerning treatment exposure, adherence, and quality of program delivery. Specifically, process data indicated that teachers did not demonstrate the same level of fidelity that was captured in classroom observations. This finding provides some evidence for the absence of an intervention effect. Major Takeaways: The current study improves our understanding of how web‐based interventions may be assessed for implementation fidelity using process data. Further, process data offers a potentially reliable and scalable measure of fidelity for other web‐based educational interventions. Lay Description: What we know about the topic: Mobile technology‐based interventions are on the rise in many educational settings.Fidelity to educational interventions can be difficult to reliably assess.Teachers may be resistant to implementing technology in classrooms. What this paper adds: Examines a pilot study of a mobile technology‐based curricular intervention for teachers designed to improve early reading and literacy.Examines a new method for assessing fidelity of mobile technology‐based interventions.Compares the new method to traditional methods of assessing fidelity to educational interventions. Implications for practice and/or policy: The presence of technology in the classroom does not assure that the technology will be used effectively.Mobile technology‐based interventions may require additional monitoring and coaching to assure use as intended.Process data, generated in the use of technology‐based interventions, may provide a unique way to measure fidelity to the intervention. pubtype: Academic Journal doctype: research tables/charts randomized controlled trial Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|