Clinical concept extraction using transformers.
Objective: The goal of this study is to explore transformer-based models (eg, Bidirectional Encoder Representations from Transformers [BERT]) for clinical concept extraction and develop an open-source package with pretrained clinical models to facilitate concept extraction and other downstream natur...
| Published in: | Journal of the American Medical Informatics Association Vol. 27; no. 12; pp. 1935 - 1943 |
|---|---|
| Main Authors: | , , , |
| Format: | research Journal Article |
| Published: |
Oxford University Press / USA
Dec2020
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=147573390&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 147573390 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 10675027 FZ9 jtl: Journal of the American Medical Informatics Association issn: 10675027 maglogo: N pubinfo: dt: Dec2020 vid: 27 iid: 12 pid: 622 pub: Oxford University Press / USA artinfo: ui: 147573390 147573390 NLM33120431 147573390 10.1093/jamia/ocaa189 NLM33120431 147573390 ppf: 1935 ppct: 8 formats: tig: atl: Clinical concept extraction using transformers. aug: au: Yang, Xi Bian, Jiang Hogan, William R Wu, Yonghui affil: Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida , Gainesville, Florida, USA sug: subj: Data Mining Methods Software Natural Language Processing Human Data Collection Comparative Studies Multicenter Studies Evaluation Research Validation Studies Short Portable Mental Status Questionnaire Scales Questionnaires ab: Objective: The goal of this study is to explore transformer-based models (eg, Bidirectional Encoder Representations from Transformers [BERT]) for clinical concept extraction and develop an open-source package with pretrained clinical models to facilitate concept extraction and other downstream natural language processing (NLP) tasks in the medical domain.Methods: We systematically explored 4 widely used transformer-based architectures, including BERT, RoBERTa, ALBERT, and ELECTRA, for extracting various types of clinical concepts using 3 public datasets from the 2010 and 2012 i2b2 challenges and the 2018 n2c2 challenge. We examined general transformer models pretrained using general English corpora as well as clinical transformer models pretrained using a clinical corpus and compared them with a long short-term memory conditional random fields (LSTM-CRFs) mode as a baseline. Furthermore, we integrated the 4 clinical transformer-based models into an open-source package.Results and Conclusion: The RoBERTa-MIMIC model achieved state-of-the-art performance on 3 public clinical concept extraction datasets with F1-scores of 0.8994, 0.8053, and 0.8907, respectively. Compared to the baseline LSTM-CRFs model, RoBERTa-MIMIC remarkably improved the F1-score by approximately 4% and 6% on the 2010 and 2012 i2b2 datasets. This study demonstrated the efficiency of transformer-based models for clinical concept extraction. Our methods and systems can be applied to other clinical tasks. The clinical transformer package with 4 pretrained clinical models is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerNER. We believe this package will improve current practice on clinical concept extraction and other tasks in the medical domain. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|