Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials.
Introduction: The automated classification of clinical trials and key categories within the medical literature is increasingly relevant, particularly in oncology, as the volume of publications and trial reports continues to expand. Large language models (LLMs) may provide new opportunities for autom...
| Publicado en: | Oncology Vol. 104; no. 4; pp. 410 - 421 |
|---|---|
| Autores principales: | , , , , , , , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Karger AG
2026
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=192909615&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 192909615 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 00302414 NK2 jtl: Oncology issn: 00302414 maglogo: N pubinfo: dt: 2026 vid: 104 iid: 4 pid: 2485 pub: Karger AG artinfo: ui: 192909615 186832399 192909615 192909615 10.1159/000546946 192909615 ppf: 410 ppct: 11 formats: tig: atl: Application of a General Large Language Model-Based Classification System to Retrieve Information about Oncological Trials. aug: au: Dennstädt, Fabio Windisch, Paul Filchenko, Irina Zink, Johannes Putora, Paul Martin Shaheen, Ahmed Gaio, Roberto Cihoric, Nikola Wosny, Marie Aeppli, Stefanie Schmerder, Max Shelan, Mohamed Hastings, Janna affil: Department of Radiation Oncology, Inselspital, Bern University Hospital and University of Bern, Bern, Switzerland sug: subj: Natural Language Processing Utilization Information Retrieval Clinical Trials Classification Neoplasms Classification Classification Algorithms Reproducibility of Results Descriptive Statistics Automation ab: Introduction: The automated classification of clinical trials and key categories within the medical literature is increasingly relevant, particularly in oncology, as the volume of publications and trial reports continues to expand. Large language models (LLMs) may provide new opportunities for automating diverse classification tasks. They could be used for general-purpose text classification, retrieving information about oncological trials. Methods: A general text classification framework with adaptable prompt, model and categories for the classification was developed. The framework was tested with four datasets comprising nine binary classification questions related to oncological trials. Evaluation was conducted using a locally hosted Mixtral-8x7B-Instruct v0.1-GPTQ model and three cloud-based LLMs: Mixtral-8x7B-Instruct v0.1, Llama3.1-70B-Instruct, and Qwen-2.5–72B. Results: The system consistently produced valid responses with the local Mixtral-8x7B-Instruct model and the Llama3.1-70B-Instruct model. It achieved a response validity rate of 99.70% and 99.88% for the cloud-based Mixtral and Qwen models, respectively. Across all models, the framework achieved an overall accuracy of >94%, precision of >92%, recall of >90%, and an F1-score of >92%. Question-specific accuracy ranged from 86.33% to 99.83% for the local Mixtral model, 85.49%–99.83% for the cloud-based Mixtral model, 90.50%–99.83% for the Llama3.1 model, and 77.13%–99.83% for the Qwen model. Conclusion: The LLM-based classification framework exhibits robust accuracy and adaptability across various oncological trial classification tasks. While there remain some challenges such as strong prompt dependence and high computational and hardware demands, LLMs will play a crucial role in automating the classification of oncological trials and literature as the technology continues to advance. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|