Performance of Natural Language Processing Model in Extracting Information from Free-Text Radiology Reports: A Systematic Review and Meta-Analysis.
The free-text format is widely used in radiology reports for its flexibility of expression; however, its unstructured nature leads to substantial amounts of report data remaining underutilized. A natural language processing (NLP) model for automatic extraction of information from free-text radiology...
| Published in: | Journal of Imaging Informatics in Medicine Vol. 39; no. 4; pp. 3639 - 3654 |
|---|---|
| Main Authors: | , , , , , , , |
| Format: | research systematic review tables/charts Journal Article |
| Published: |
Springer Nature
Aug2026
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=196241804&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 196241804 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 29482925 NR3A jtl: Journal of Imaging Informatics in Medicine issn: 29482925 maglogo: N pubinfo: dt: Aug2026 vid: 39 iid: 4 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 196241804 190286238 196241804 196241804 10.1007/s10278-025-01728-8 196241804 ppf: 3639 ppct: 15 formats: tig: atl: Performance of Natural Language Processing Model in Extracting Information from Free-Text Radiology Reports: A Systematic Review and Meta-Analysis. aug: au: Yang, Qingwen Jiang, Jiahui Dong, Xue Yang, Huai Wang, Qiuren Yang, Zhenghan Yang, Dawei Liu, Peng affil: https://ror.org/013xs5b60 Department of Radiology, Beijing Friendship Hospital, Capital Medical University, No. 95 Yong'an Road, Xicheng District, 100050, Beijing, China sug: subj: Information Retrieval Natural Language Processing Utilization Electronic Health Records Reports Radiology Information Systems Human Systematic Review Meta Analysis PubMed Medline Embase Cochrane Library Scales Sensitivity and Specificity Confidence Intervals Maximum Likelihood Regression ROC Curve Data Analysis Software Descriptive Statistics Funding Source ab: The free-text format is widely used in radiology reports for its flexibility of expression; however, its unstructured nature leads to substantial amounts of report data remaining underutilized. A natural language processing (NLP) model for automatic extraction of information from free-text radiology reports can significantly contribute to the development of structured databases, thereby optimizing data utilization. This study aimed to perform a systematic review and meta-analysis that evaluates the performance of NLP systems in extracting information from free-text radiology reports. A systematic literature search was conducted from November 21 to 23, 2024, in PubMed/MEDLINE, Embase, EBSCO, Ovid, Web of Science, and the Cochrane Library. Study quality was assessed using the QUADAS-2 tool. A bivariate random-effects model was applied to obtain the pooled sensitivity, specificity, diagnostic odds ratio (DOR), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and area under the summary receiver operating characteristic curve (AUC). Subgroup analyses (e.g., NLP model types, dataset source, and language types) and a random-effects multivariable meta-regression based on the restricted maximum likelihood (REML) method were conducted to explore potential sources of heterogeneity. Sensitivity analyses (excluding high-risk studies, leave-one-out method, and data integration strategy comparison) were performed to assess the robustness of the findings. A total of 28 studies were included in the final analysis, with 421,692 extracted entities in 51,187 free-text radiology reports. NLP systems achieved high pooled sensitivity (91% [95% CI: 87, 93]) and specificity (96% [95% CI: 93, 97]), with a diagnostic odds ratio of 220 (95% CI: 112, 435) and an area under the curve of 0.98 (95% CI: 0.96, 0.99). Subgroup analysis revealed significantly better performance for extracting single anatomical sites (AUC 0.99; 95% CI: 0.97, 0.99) compared with multiple sites (AUC 0.95; 95% CI: 0.93, 0.97; p = 0.001). No significant differences were observed across NLP model types, dataset sources, external validations, languages, or imaging modalities. Multivariable meta-regression further identified anatomical site as the only significant contributor to heterogeneity (coefficient = 2.26; 95% CI: 0.25, 4.27; p = 0.027). Sensitivity analyses confirmed the robustness of the findings, and no evidence of publication bias was detected. NLP models demonstrated excellent performance in extracting information from free-text radiology reports. However, the observed heterogeneity highlights the need for enhanced report standardization and improved model generalizability. pubtype: Academic Journal doctype: meta analysis research systematic review tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|