Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning.
Background: Machine learning approaches have become increasingly popular modeling techniques, relying on data-driven heuristics to arrive at its solutions. Recent comparisons between these algorithms and traditional statistical modeling techniques have largely ignored the superiority gained by the f...
| Published in: | BMC Medical Research Methodology Vol. 20; no. 1; pp. 1 - 16 |
|---|---|
| Main Authors: | , |
| Format: | research Journal Article |
| Published: |
BioMed Central
6/29/2020
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=144295931&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 144295931 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 14712288 1CI1 jtl: BMC Medical Research Methodology issn: 14712288 maglogo: N pubinfo: dt: 6/29/2020 vid: 20 iid: 1 pid: 24147 pub: BioMed Central artinfo: ui: 144295931 144295931 NLM32600277 144295931 10.1186/s12874-020-01046-3 NLM32600277 144295931 ppf: 1 ppct: 15 formats: tig: atl: Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning. aug: au: Levy, Joshua J. O'Malley, A. James affil: Program in Quantitative Biomedical Sciences, Geisel School of Medicine at Dartmouth, Hanover, USA sug: subj: Models, Statistical Algorithms Human Logistic Regression Comparative Studies Multicenter Studies Evaluation Research Validation Studies Questionnaires ab: Background: Machine learning approaches have become increasingly popular modeling techniques, relying on data-driven heuristics to arrive at its solutions. Recent comparisons between these algorithms and traditional statistical modeling techniques have largely ignored the superiority gained by the former approaches due to involvement of model-building search algorithms. This has led to alignment of statistical and machine learning approaches with different types of problems and the under-development of procedures that combine their attributes. In this context, we hoped to understand the domains of applicability for each approach and to identify areas where a marriage between the two approaches is warranted. We then sought to develop a hybrid statistical-machine learning procedure with the best attributes of each.Methods: We present three simple examples to illustrate when to use each modeling approach and posit a general framework for combining them into an enhanced logistic regression model building procedure that aids interpretation. We study 556 benchmark machine learning datasets to uncover when machine learning techniques outperformed rudimentary logistic regression models and so are potentially well-equipped to enhance them. We illustrate a software package, InteractionTransformer, which embeds logistic regression with advanced model building capacity by using machine learning algorithms to extract candidate interaction features from a random forest model for inclusion in the model. Finally, we apply our enhanced logistic regression analysis to two real-word biomedical examples, one where predictors vary linearly with the outcome and another with extensive second-order interactions.Results: Preliminary statistical analysis demonstrated that across 556 benchmark datasets, the random forest approach significantly outperformed the logistic regression approach. We found a statistically significant increase in predictive performance when using hybrid procedures and greater clarity in the association with the outcome of terms acquired compared to directly interpreting the random forest output.Conclusions: When a random forest model is closer to the true model, hybrid statistical-machine learning procedures can substantially enhance the performance of statistical procedures in an automated manner while preserving easy interpretation of the results. Such hybrid methods may help facilitate widespread adoption of machine learning techniques in the biomedical setting. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|