Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error.
Background: Retrospective exploratory analyses of randomised controlled trials (RCTs) seeking to identify treatment effect heterogeneity (TEH) are prone to bias and false positives. Yet the desire to learn all we can from exhaustive data measurements on trial participants motivates the inclusion of...
| Publicado en: | Trials Vol. 21; no. 1; pp. 1 - 11 |
|---|---|
| Autores principales: | , |
| Formato: | equations & formulas research tables/charts Journal Article |
| Publicado: |
BioMed Central
2/10/2020
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=141662396&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 141662396 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 17456215 38OC jtl: Trials issn: 17456215 maglogo: N pubinfo: dt: 2/10/2020 vid: 21 iid: 1 pid: 24147 pub: BioMed Central artinfo: ui: 141662396 141662396 NLM32041653 141662396 10.1186/s13063-020-4076-y NLM32041653 141662396 ppf: 1 ppct: 10 formats: tig: atl: Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error. aug: au: Watson, James A. Holmes, Chris C. affil: Mahidol Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University, Rajvithi Road, 10400, Bangkok, Thailand sug: subj: Antimalarials Therapeutic Use Protozoa Drug Effects Clinical Trials Quinine Therapeutic Use Malaria Drug Therapy Treatment Effect Heterogeneity Algorithms Asia Adult Treatment Outcomes Malaria Retrospective Design Malaria Epidemiology Funding Source Adult: 19-44 years ab: Background: Retrospective exploratory analyses of randomised controlled trials (RCTs) seeking to identify treatment effect heterogeneity (TEH) are prone to bias and false positives. Yet the desire to learn all we can from exhaustive data measurements on trial participants motivates the inclusion of such analyses within RCTs. Moreover, widespread advances in machine learning (ML) methods hold potential to utilise such data to identify subjects exhibiting heterogeneous treatment response.Methods: We present a novel analysis strategy for detecting TEH in randomised data using ML methods, whilst ensuring proper control of the false positive discovery rate. Our approach uses random data partitioning with statistical or ML-based prediction on held-out data. This method can test for both crossover TEH (switch in optimal treatment) and non-crossover TEH (systematic variation in benefit across patients). The former is done via a two-sample hypothesis test measuring overall predictive performance. The latter is done via 'stacking' the ML predictors alongside a classical statistical model to formally test the added benefit of the ML algorithm. An adaptation of recent statistical theory allows for the construction of a valid aggregate p value. This testing strategy is independent of the choice of ML method.Results: We demonstrate our approach with a re-analysis of the SEAQUAMAT trial, which compared quinine to artesunate for the treatment of severe malaria in Asian adults. We find no evidence for any subgroup who would benefit from a change in treatment from the current standard of care, artesunate, but strong evidence for significant TEH within the artesunate treatment group. In particular, we find that artesunate provides a differential benefit to patients with high numbers of circulating ring stage parasites.Conclusions: ML analysis plans using computational notebooks (documents linked to a programming language that capture the model parameter settings, data processing choices, and evaluation criteria) along with version control can improve the robustness and transparency of RCT exploratory analyses. A data-partitioning algorithm allows researchers to apply the latest ML techniques safe in the knowledge that any declared associations are statistically significant at a user-defined level. pubtype: Academic Journal doctype: equations & formulas research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|