Assessment and Integration of Large Language Models for Automated Electronic Health Record Documentation in Emergency Medical Services.
Automating Electronic Health Records (EHR) documentation can significantly reduce the burden on care providers, particularly in emergency care settings where rapid and accurate record-keeping is crucial. A critical aspect of this automation involves using natural language processing (NLP) techniques...
| Publicado en: | Journal of Medical Systems Vol. 49; no. 1; pp. 1 - 20 |
|---|---|
| Autores principales: | , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Springer Nature
5/17/2025
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=185239820&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 185239820 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 01485598 4N0 jtl: Journal of Medical Systems issn: 01485598 maglogo: N pubinfo: dt: 5/17/2025 vid: 49 iid: 1 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 185239820 185239820 185239820 10.1007/s10916-025-02197-w 185239820 ppf: 1 ppct: 19 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: Assessment and Integration of Large Language Models for Automated Electronic Health Record Documentation in Emergency Medical Services. aug: au: Bai, Enze Luo, Xiao Zhang, Zhan Adelgais, Kathleen Ali, Humaira Finkelstein, Jack Kutzin, Jared affil: https://ror.org/047p7y759 School of Computer Science and Information Systems, Pace University, New York City, NY, USA sug: subj: Natural Language Processing Evaluation Electronic Health Records Documentation Automation Emergency Medical Services Human Emergency Service Information Systems Program Implementation Program Development Descriptive Statistics Comparative Studies Funding Source ab: Automating Electronic Health Records (EHR) documentation can significantly reduce the burden on care providers, particularly in emergency care settings where rapid and accurate record-keeping is crucial. A critical aspect of this automation involves using natural language processing (NLP) techniques to convert transcribed conversations into structured EHR fields. For instance, extracting temperature values like "102.4 Fahrenheit" from the transcribed text "His temperature is 39.1, which is 102.4 Fahrenheit." However, traditional rule-based and single-model NLP approaches often struggle with domain-specific medical terminology, contextual ambiguity, and numerical extraction errors. This study investigates the potential of integrating multiple Large Language Models (LLMs) to enhance EMS documentation accuracy. We developed an LLM integration framework and evaluated four state-of-the-art LLMs—Claude 3.5, GPT-4, Gemini, and Mistral—on a dataset comprising transcribed conversations from 40 EMS training simulations. The evaluation focused on precision, recall, and F1 score across zero-shot and few-shot learning scenarios. Results showed that the integrated LLM framework outperformed individual models, achieving overall F1 scores of 0.78 (zero-shot) and 0.81 (few-shot). In addition to quantitative evaluation, a preliminary user study was conducted with domain experts to assess the perceived usefulness and challenges of the integrated framework. The findings suggest that this approach has the potential to reduce documentation effort compared to traditional manual documentation. However, challenges such as misinterpretation of medical context and occasional omissions were noted, highlighting areas for further refinement and future work. This research is the first to systematically explore and evaluate the use of LLMs for real-time EMS EHR documentation. By addressing key challenges in automated transcription and structured data extraction, our work lays a foundation for real-world implementation, improving efficiency and accuracy in emergency medical documentation. Clinical trial number: Not applicable. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|