The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports.
Early detection of patients with impending bone metastasis is crucial for prognosis improvement. This study aimed to investigate the feasibility of a fine-tuned, locally run large language model (LLM) in extracting patients with bone metastasis in unstructured Japanese radiology report and to compar...
| Published in: | Journal of Imaging Informatics in Medicine Vol. 38; no. 2; pp. 865 - 873 |
|---|---|
| Main Authors: | , , , , |
| Format: | research tables/charts Journal Article |
| Published: |
Springer Nature
Apr2025
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=184081745&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 184081745 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 29482925 NR3A jtl: Journal of Imaging Informatics in Medicine issn: 29482925 maglogo: N pubinfo: dt: Apr2025 vid: 38 iid: 2 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 184081745 184081745 184081745 10.1007/s10278-024-01242-3 184081745 ppf: 865 ppct: 8 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports. aug: au: Kanemaru, Noriko Yasaka, Koichiro Fujita, Nana Kanzawa, Jun Abe, Osamu affil: https://ror.org/057zh3y96 Department of Radiology, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, 113-8655, Tokyo, Japan sug: subj: Natural Language Processing Bone Metastases Reports Japan Human Funding Source Japan Pilot Studies Retrospective Design Record Review Sensitivity and Specificity Deep Learning Data Analysis Software Descriptive Statistics Chi Square Test Male Female Male Female ab: Early detection of patients with impending bone metastasis is crucial for prognosis improvement. This study aimed to investigate the feasibility of a fine-tuned, locally run large language model (LLM) in extracting patients with bone metastasis in unstructured Japanese radiology report and to compare its performance with manual annotation. This retrospective study included patients with "metastasis" in radiological reports (April 2018–January 2019, August–May 2022, and April–December 2023 for training, validation, and test datasets of 9559, 1498, and 7399 patients, respectively). Radiologists reviewed the clinical indication and diagnosis sections of the radiological report (used as input data) and classified them into groups 0 (no bone metastasis), 1 (progressive bone metastasis), and 2 (stable or decreased bone metastasis). The data for group 0 was under-sampled in training and test datasets due to group imbalance. The best-performing model from the validation set was subsequently tested using the testing dataset. Two additional radiologists (readers 1 and 2) were involved in classifying radiological reports within the test dataset for testing purposes. The fine-tuned LLM, reader 1, and reader 2 demonstrated an accuracy of 0.979, 0.996, and 0.993, sensitivity for groups 0/1/2 of 0.988/0.947/0.943, 1.000/1.000/0.966, and 1.000/0.982/0.954, and time required for classification (s) of 105, 2312, and 3094 in under-sampled test dataset (n = 711), respectively. Fine-tuned LLM extracted patients with bone metastasis, demonstrating satisfactory performance that was comparable to or slightly lower than manual annotation by radiologists in a noticeably shorter time. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|