Modification and Validation of the System Causability Scale Using AI-Based Therapeutic Recommendations for Urological Cancer Patients: A Basis for the Development of a Prospective Comparative Study.
The integration of artificial intelligence, particularly Large Language Models (LLMs), has the potential to significantly enhance therapeutic decision-making in clinical oncology. Initial studies across various disciplines have demonstrated that LLM-based treatment recommendations can rival those of...
| Published in: | Current Oncology Vol. 31; no. 11; pp. 7061 - 7074 |
|---|---|
| Main Authors: | , , , , , , , , , , , , , , , , , , |
| Format: | Journal Article |
| Published: |
MDPI
Nov2024
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=181170881&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 181170881 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 11980052 5EKK jtl: Current Oncology issn: 11980052 maglogo: N pubinfo: dt: Nov2024 vid: 31 iid: 11 pid: 97109 pub: MDPI artinfo: ui: 181170881 10.3390/curroncol31110520 181170881 ppf: 7061 ppct: 13 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: Modification and Validation of the System Causability Scale Using AI-Based Therapeutic Recommendations for Urological Cancer Patients: A Basis for the Development of a Prospective Comparative Study. aug: au: Rinderknecht, Emily von Winning, Dominik Kravchuk, Anton Schäfer, Christof Schnabel, Marco J. Siepmann, Stephan Mayr, Roman Grassinger, Jochen Goßler, Christopher Pohl, Fabian Siska, Peter J. Zeman, Florian Breyer, Johannes Schmelzer, Anna Gilfrich, Christian Brookman-May, Sabine D. Burger, Maximilian Haas, Maximilian May, Matthias affil: Department of Urology, Caritas St. Josef Hospital, University of Regensburg,93053 Regensburg, Germany sug: ab: The integration of artificial intelligence, particularly Large Language Models (LLMs), has the potential to significantly enhance therapeutic decision-making in clinical oncology. Initial studies across various disciplines have demonstrated that LLM-based treatment recommendations can rival those of multidisciplinary tumor boards (MTBs); however, such data are currently lacking for urological cancers. This preparatory study establishes a robust methodological foundation for the forthcoming CONCORDIA trial, including the validation of the System Causability Scale (SCS) and its modified version (mSCS), as well as the selection of LLMs for urological cancer treatment recommendations based on recommendations from ChatGPT-4 and an MTB for 40 urological cancer scenarios. Both scales demonstrated strong validity, reliability (all aggregated Cohen's K > 0.74), and internal consistency (all Cronbach's Alpha > 0.9), with the mSCS showing superior reliability, internal consistency, and clinical applicability (p < 0.01). Two Delphi processes were used to define the LLMs to be tested in the CONCORDIA study (ChatGPT-4 and Claude 3.5 Sonnet) and to establish the acceptable non-inferiority margin for LLM recommendations compared to MTB recommendations. The forthcoming ethics-approved and registered CONCORDIA non-inferiority trial will require 110 urological cancer scenarios, with an mSCS difference threshold of 0.15, a Bonferroni corrected alpha of 0.025, and a beta of 0.1. Blinded mSCS assessments of MTB recommendations will then be compared to those of the LLMs. In summary, this work establishes the necessary prerequisites prior to initiating the CONCORDIA study and validates a modified score with high applicability and reliability for this and future trials. pubtype: Academic Journal doctype: Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|