External validation of a machine learning classifier to identify unhealthy alcohol use in hospitalized patients.
Background and Aims: Unhealthy alcohol use (UAU) is one of the leading causes of global morbidity. A machine learning approach to alcohol screening could accelerate best practices when integrated into electronic health record (EHR) systems. This study aimed to validate externally a natural language...
| Publicado en: | Addiction Vol. 117; no. 4; pp. 925 - 934 |
|---|---|
| Autores principales: | , , , , , , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Apr2022
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=155656573&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 155656573 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 09652140 AIO jtl: Addiction issn: 09652140 maglogo: Y pubinfo: dt: Apr2022 vid: 117 iid: 4 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 155656573 153729176 155656573 155656573 10.1111/add.15730 155656573 ppf: 925 ppct: 9 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: External validation of a machine learning classifier to identify unhealthy alcohol use in hospitalized patients. aug: au: Lin, Yiqi Sharma, Brihat Thompson, Hale M. Boley, Randy Perticone, Kathryn Chhabra, Neeraj Afshar, Majid Karnik, Niranjan S. affil: Rush Medical College, Rush University, Chicago IL,, USA sug: subj: Hospitalization Alcohol Abuse Classification Machine Learning Utilization Human External Validity Natural Language Processing Inpatients Retrospective Design Prospective Studies Midwestern United States Electronic Health Records Documentation Clinical Assessment Tools External Validity Evaluation Female Male ROC Curve Confidence Intervals Predictive Value of Tests Sensitivity and Specificity Data Science Female Male ab: Background and Aims: Unhealthy alcohol use (UAU) is one of the leading causes of global morbidity. A machine learning approach to alcohol screening could accelerate best practices when integrated into electronic health record (EHR) systems. This study aimed to validate externally a natural language processing (NLP) classifier developed at an independent medical center. Design Retrospective cohort study. Setting: The site for validation was a midwestern United States tertiary‐care, urban medical center that has an inpatient structured universal screening model for unhealthy substance use and an active addiction consult service. Participants/Cases: Unplanned admissions of adult patients between October 23, 2017 and December 31, 2019, with EHR documentation of manual alcohol screening were included in the cohort (n = 57 605). Measurements The Alcohol Use Disorders Identification Test (AUDIT) served as the reference standard. AUDIT scores ≥5 for females and ≥8 for males served as cases for UAU. To examine error in manual screening or under‐reporting, a post hoc error analysis was conducted, reviewing discordance between the NLP classifier and AUDIT‐derived reference. All clinical notes excluding the manual screening and AUDIT documentation from the EHR were included in the NLP analysis. Findings Using clinical notes from the first 24 hours of each encounter, the NLP classifier demonstrated an area under the receiver operating characteristic curve (AUCROC) and precision‐recall area under the curve (PRAUC) of 0.91 (95% CI = 0.89–0.92) and 0.56 (95% CI = 0.53–0.60), respectively. At the optimal cut point of 0.5, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 0.66 (95% CI = 0.62–0.69), 0.98 (95% CI = 0.98–0.98), 0.35 (95% CI = 0.33–0.38), and 1.0 (95% CI = 1.0–1.0), respectively. Conclusions: External validation of a publicly available alcohol misuse classifier demonstrates adequate sensitivity and specificity for routine clinical use as an automated screening tool for identifying at‐risk patients. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|