Evaluation of clinical named entity recognition methods for Serbian electronic health records.
Background and Objectives: The importance of clinical natural language processing (NLP) has increased with the adoption of electronic health records (EHRs). One of the critical tasks in clinical NLP is named entity recognition (NER). Clinical NER in the Serbian language is a severely under-researche...
| Published in: | International Journal of Medical Informatics Vol. 164 |
|---|---|
| Main Authors: | , , , , , |
| Format: | research Journal Article |
| Published: |
Elsevier B.V.
Aug2022
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=157393407&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 157393407 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 13865056 JR4 jtl: International Journal of Medical Informatics issn: 13865056 maglogo: N pubinfo: dt: Aug2022 vid: 164 pid: 467 pub: Elsevier B.V. place: New York, New York artinfo: ui: 157393407 157393407 NLM35653828 157393407 10.1016/j.ijmedinf.2022.104805 NLM35653828 157393407 ppct: 1 formats: tig: atl: Evaluation of clinical named entity recognition methods for Serbian electronic health records. aug: au: Kaplar, Aleksandar Stošović, Milan Kaplar, Aleksandra Brković, Voin Naumović, Radomir Kovačević, Aleksandar affil: Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia sug: subj: Natural Language Processing Serbia Human ab: Background and Objectives: The importance of clinical natural language processing (NLP) has increased with the adoption of electronic health records (EHRs). One of the critical tasks in clinical NLP is named entity recognition (NER). Clinical NER in the Serbian language is a severely under-researched area. The few approaches that have been proposed so far are based on rules or machine-learning models with hand-crafted features, while current state-of-the-art models have not been explored. The objective of this paper is to assess the performance of state-of-the-art NER methods on clinical narratives in the Serbian language.Materials and Methods: We designed an experimental setup for a comprehensive evaluation of state-of-the-art NER models. The gold standard corpus we used for the evaluation is comprised of discharge summaries from the Clinic for Nephrology at the University Clinical Center of Serbia. The following models were evaluated: conditional random fields (CRF), multilingual transformers (BERT Multilingual and XLM RoBERTa), and long short-term memory (LSTM) recurrent neural networks, and their ensembles. In addition, we investigated the necessity of the pretraining task of transformer based models and the use of pretrained word embeddings with LSTM model.Results: Our results show that individually CRF had the best precision, the pretrained BERT Multilingual model had the best recall values, and the LSTM model had the best F1 score. The best performance was achieved by combining the existing models in a majority voting ensemble with an F1 score of 0.892. The presented results are similar to the inter annotator agreement on our gold standard corpus and are comparable to existing state-of-the-art results for clinical NER reported in literature.Conclusion: Existing state-of-the-art models can provide viable results for clinical named entity recognition when applied to languages with the complexity of the Serbian language without major modifications. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|