Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation.
Objective: Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementat...
| Publicado en: | Emergency Medicine Australasia Vol. 36; no. 1; pp. 118 - 125 |
|---|---|
| Autores principales: | , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Feb2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=174818049&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 174818049 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 17426731 Z3W jtl: Emergency Medicine Australasia issn: 17426731 maglogo: Y pubinfo: dt: Feb2024 vid: 36 iid: 1 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 174818049 172374996 174818049 174818049 10.1111/1742-6723.14325 174818049 ppf: 118 ppct: 7 formats: tig: atl: Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation. aug: au: Akhlaghi, Hamed Freeman, Sam Vari, Cynthia McKenna, Bede Braitberg, George Karro, Jonathan Tahayori, Bahman affil: Department of Emergency Medicine, St Vincent's Hospital Melbourne, Melbourne Victoria,, Australia sug: subj: Machine Learning Methods Artificial Intelligence Methods Gastroenterology Care Clinical Competence Implementation Science Clinical Assessment Tools Evaluation Human Male Female Hospitalization Algorithms Health Care Delivery Triage Predictive Value of Tests Decision Support Systems, Clinical Program Evaluation Decision Making, Computer Assisted Health Personnel Psychosocial Factors Descriptive Statistics Comparative Studies Confidence Intervals Male Female ab: Objective: Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementation. This is the first of such studies to investigate real‐time AI performance in the emergency setting. Methods: The novel AI algorithm that predicts admission using a triage note was translated into clinical practice and integrated within St Vincent's Hospital Melbourne's electronic emergency patient management system. The data were collected from 1 January 2021 to 17 August 2022 to evaluate the diagnostic accuracy of the AI system after implementation. Results: A total of 77 125 ED presentations were included. The live AI algorithm has a sensitivity of 73.1% (95% confidence interval 72.5–73.8), specificity of 74.3% (73.9–74.7), positive predictive value of 50% (49.6–50.4) and negative predictive value of 88.7% (88.5–89) with a total accuracy of 74% (73.7–74.3). The accuracy of the system was at the lowest for admission to psychiatric units (34%) and at the highest for gastroenterology and medical admission (84% and 80%, respectively). Conclusion: Our study showed the diagnostic evaluation of a real‐time AI clinical decision‐support tool became less accurate than the original. Although real‐time sensitivity and specificity of the AI tool was still acceptable as a decision‐support tool in the ED, we propose that continuous training and evaluation of AI‐enabled clinical support tools in healthcare are conducted to ensure consistent accuracy and performance to prevent inadvertent consequences. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|