Outcome class imbalance and rare events: An underappreciated complication for overdose risk prediction modeling.
Background and aims: Low outcome prevalence, often observed with opioid‐related outcomes, poses an underappreciated challenge to accurate predictive modeling. Outcome class imbalance, where non‐events (i.e. negative class observations) outnumber events (i.e. positive class observations) by a moderat...
| Publicado en: | Addiction Vol. 118; no. 6; pp. 1167 - 1177 |
|---|---|
| Autores principales: | , , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Jun2023
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=163589549&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 163589549 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 09652140 AIO jtl: Addiction issn: 09652140 maglogo: Y pubinfo: dt: Jun2023 vid: 118 iid: 6 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 163589549 161709262 163589549 163589549 10.1111/add.16133 163589549 ppf: 1167 ppct: 10 formats: fmt: – @attributes: type: T – @attributes: type: C – @attributes: type: P tig: atl: Outcome class imbalance and rare events: An underappreciated complication for overdose risk prediction modeling. aug: au: Cartus, Abigail R. Samuels, Elizabeth A. Cerdá, Magdalena Marshall, Brandon D. L. affil: Department of Epidemiology, Brown University School of Public Health, Providence Rhode Island,, USA sug: subj: Analgesics, Opioid Adverse Effects Overdose Risk Factors Outcome Assessment Risk Assessment Prediction Models Human Convenience Sample Literature Review Algorithms Clinical Assessment Tools Funding Source ab: Background and aims: Low outcome prevalence, often observed with opioid‐related outcomes, poses an underappreciated challenge to accurate predictive modeling. Outcome class imbalance, where non‐events (i.e. negative class observations) outnumber events (i.e. positive class observations) by a moderate to extreme degree, can distort measures of predictive accuracy in misleading ways, and make the overall predictive accuracy and the discriminatory ability of a predictive model appear spuriously high. We conducted a simulation study to measure the impact of outcome class imbalance on predictive performance of a simple SuperLearner ensemble model and suggest strategies for reducing that impact. Design, Setting, Participants: Using a Monte Carlo design with 250 repetitions, we trained and evaluated these models on four simulated data sets with 100 000 observations each: one with perfect balance between events and non‐events, and three where non‐events outnumbered events by an approximate factor of 10:1, 100:1, and 1000:1, respectively. Measurements: We evaluated the performance of these models using a comprehensive suite of measures, including measures that are more appropriate for imbalanced data. Findings Increasing imbalance tended to spuriously improve overall accuracy (using a high threshold to classify events vs non‐events, overall accuracy improved from 0.45 with perfect balance to 0.99 with the most severe outcome class imbalance), but diminished predictive performance was evident using other metrics (corresponding positive predictive value decreased from 0.99 to 0.14). Conclusion: Increasing reliance on algorithmic risk scores in consequential decision‐making processes raises critical fairness and ethical concerns. This paper provides broad guidance for analytic strategies that clinical investigators can use to remedy the impacts of outcome class imbalance on risk prediction tools. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|