Improved Fine-Tuning of In-Domain Transformer Model for Inferring COVID-19 Presence in Multi-Institutional Radiology Reports.
Building a document-level classifier for COVID-19 on radiology reports could help assist providers in their daily clinical routine, as well as create large numbers of labels for computer vision models. We have developed such a classifier by fine-tuning a BERT-like model initialized from RadBERT, its...
| Publicado en: | Journal of Digital Imaging Vol. 36; no. 1; pp. 164 - 178 |
|---|---|
| Autores principales: | , , |
| Formato: | equations & formulas research tables/charts Journal Article |
| Publicado: |
Springer Nature
Feb2023
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=162233265&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 162233265 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 08971889 DOQ jtl: Journal of Digital Imaging issn: 08971889 maglogo: N pubinfo: dt: Feb2023 vid: 36 iid: 1 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 162233265 159997437 162233265 162233265 10.1007/s10278-022-00714-8 162233265 ppf: 164 ppct: 14 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: Improved Fine-Tuning of In-Domain Transformer Model for Inferring COVID-19 Presence in Multi-Institutional Radiology Reports. aug: au: Chambon, Pierre Cook, Tessa S. Langlotz, Curtis P. affil: Stanford University, Paris-Saclay University, École Centrale Paris, Stanford, USA sug: subj: COVID-19 Radiography COVID-19 Classification Natural Language Processing Radiographic Image Enhancement Radiographic Image Interpretation, Computer-Assisted Human Multicenter Studies Radiography, Thoracic Tomography, X-Ray Computed Lung Diseases Radiography ab: Building a document-level classifier for COVID-19 on radiology reports could help assist providers in their daily clinical routine, as well as create large numbers of labels for computer vision models. We have developed such a classifier by fine-tuning a BERT-like model initialized from RadBERT, its continuous pre-training on radiology reports that can be used on all radiology-related tasks. RadBERT outperforms all biomedical pre-trainings on this COVID-19 task (P<0.01) and helps our fine-tuned model achieve an 88.9 macro-averaged F1-score, when evaluated on both X-ray and CT reports. To build this model, we rely on a multi-institutional dataset re-sampled and enriched with concurrent lung diseases, helping the model to resist to distribution shifts. In addition, we explore a variety of fine-tuning and hyperparameter optimization techniques that accelerate fine-tuning convergence, stabilize performance, and improve accuracy, especially when data or computational resources are limited. Finally, we provide a set of visualization tools and explainability methods to better understand the performance of the model, and support its practical use in the clinical setting. Our approach offers a ready-to-use COVID-19 classifier and can be applied similarly to other radiology report classification tasks. pubtype: Academic Journal doctype: equations & formulas research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|