Duration versus accuracy—what matters for computerised adaptive testing in schools?
Background: Computerised adaptive tests (CATs) are tests that provide personalised, efficient and accurate measurement while reducing testing time, depending on the desired level of precision. Schools have different types of assessments that can benefit from a significant reduction in testing time t...
| Publicado en: | Journal of Computer Assisted Learning Vol. 40; no. 6; pp. 3443 - 3454 |
|---|---|
| Autores principales: | , , , |
| Formato: | algorithm research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Dec2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=180899703&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 180899703 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Dec2024 vid: 40 iid: 6 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 180899703 180102027 180899703 180899703 10.1111/jcal.13074 180899703 ppf: 3443 ppct: 11 formats: tig: atl: Duration versus accuracy—what matters for computerised adaptive testing in schools? aug: au: Ebenbeck, Nikola Bastian, Morten Mühling, Andreas Gebhardt, Markus affil: Special Educational Needs in Learning, Ludwig‐Maximilians‐University of Munich, München, Germany sug: subj: Computerized Adaptive Testing Student Performance Appraisal Schools Germany Reading Simulations Students with Disabilities Education, Special Human Schools, Elementary Schools, Special Germany Funding Source Intellectual Disability Female Male Language Processing Language Tests Descriptive Statistics Female Male ab: Background: Computerised adaptive tests (CATs) are tests that provide personalised, efficient and accurate measurement while reducing testing time, depending on the desired level of precision. Schools have different types of assessments that can benefit from a significant reduction in testing time to varying degrees, depending on the area of application, but for which the loss of measurement accuracy has a different impact. The implementation of CAT can take several approaches, each of which can potentially affect the resulting test length and accuracy. Objectives: We compare the methods of estimation‐based CAT and binary‐search‐based CAT to determine the extent to which they are suitable for school assessment in terms of their length and accuracy. Methods: This study uses simulations based on empirical data from a cohort of pupils with and without special needs (n = 400) to examine the effects of probabilistic estimation‐based CAT and deterministic binary‐search‐based CAT on the length and accuracy of an adaptive reading test for pupils with different ability levels. Results and Conclusions: Estimation‐based CAT leads to a 40% test reduction with an average accuracy of r = 0.96, while binary‐search‐based CAT leads to a test reduction of up to 88% with an average accuracy of r = 0.83. Both methods demonstrate the applicability of CAT in educational environments. Practical advantages and disadvantages of both methods for learning environments are discussed, as well as which method is best suited for specific assessment needs. Lay Description: What is currently known about this?Computerised adaptive testing (CAT) can significantly reduce the length of testing instruments while still accurately measuring an individual's ability.Pupils with low ability benefit most from CAT.There are several methodological approaches to adaptive testing, each with different advantages and disadvantages in terms of test duration and accuracy. What does this paper add?Estimation‐based CAT is more accurate than binary‐search‐based CAT, but requires more items.Binary search‐based CAT is suitable for short‐term and recurrent allocation of support groups or for measuring learning progress with a large increase.Estimation‐based CAT is also suitable for measures that have long‐term implications. Implications for practice:CAT can be very helpful in data‐driven decision making in schools, as it measures and automatically scores students with different levels of ability in a time‐ and resource‐efficient and individualised way.When embedded in digital learning environments, it provides a useful synthesis of assessment, teaching and learning. pubtype: Academic Journal doctype: algorithm research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|