Generalization of Deep Learning in Digital Pathology: Experience in Breast Cancer Metastasis Detection.
Simple Summary: Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer mode...
| Publicado en: | Cancers Vol. 14; no. 21; pp. 5424 - 5442 |
|---|---|
| Autores principales: | , , , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
MDPI
Nov2022
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=160147462&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 160147462 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 20726694 B74B jtl: Cancers issn: 20726694 maglogo: N pubinfo: dt: Nov2022 vid: 14 iid: 21 pid: 97109 pub: MDPI artinfo: ui: 160147462 160147462 160147462 10.3390/cancers14215424 160147462 ppf: 5424 ppct: 18 formats: tig: atl: Generalization of Deep Learning in Digital Pathology: Experience in Breast Cancer Metastasis Detection. aug: au: Jarkman, Sofia Karlberg, Micael Pocevičiūtė, Milda Bodén, Anna Bándi, Péter Litjens, Geert Lundström, Claes Treanor, Darren van der Laak, Jeroen affil: Department of Clinical Pathology, and Department of Biomedical and Clinical Sciences, Linköping University, 581 83 Linköping, Sweden sug: subj: Deep Learning Pathology, Clinical Breast Neoplasms Diagnosis Neoplasm Metastasis Diagnosis Lymph Nodes Pathology Lymphatic Diseases Diagnosis Cancer Screening Human Artificial Intelligence Multicenter Studies Descriptive Statistics Confidence Intervals Qualitative Studies Pathologists ab: Simple Summary: Pathology is a cornerstone in cancer diagnostics, and digital pathology and artificial intelligence-driven image analysis could potentially save time and enhance diagnostic accuracy. For clinical implementation of artificial intelligence, a major question is whether the computer models maintain high performance when applied to new settings. We tested the generalizability of a highly accurate deep learning model for breast cancer metastasis detection in sentinel lymph nodes from, firstly, unseen sentinel node data and, secondly, data with a small change in surgical indication, in this case lymph nodes from axillary dissections. Model performance dropped in both settings, particularly on axillary dissection nodes. Retraining of the model was needed to mitigate the performance drop. The study highlights the generalization challenge of clinical implementation of AI models, and the possibility that retraining might be necessary. Poor generalizability is a major barrier to clinical implementation of artificial intelligence in digital pathology. The aim of this study was to test the generalizability of a pretrained deep learning model to a new diagnostic setting and to a small change in surgical indication. A deep learning model for breast cancer metastases detection in sentinel lymph nodes, trained on CAMELYON multicenter data, was used as a base model, and achieved an AUC of 0.969 (95% CI 0.926–0.998) and FROC of 0.838 (95% CI 0.757–0.913) on CAMELYON16 test data. On local sentinel node data, the base model performance dropped to AUC 0.929 (95% CI 0.800–0.998) and FROC 0.744 (95% CI 0.566–0.912). On data with a change in surgical indication (axillary dissections) the base model performance indicated an even larger drop with a FROC of 0.503 (95%CI 0.201–0.911). The model was retrained with addition of local data, resulting in about a 4% increase for both AUC and FROC for sentinel nodes, and an increase of 11% in AUC and 49% in FROC for axillary nodes. Pathologist qualitative evaluation of the retrained model´s output showed no missed positive slides. False positives, false negatives and one previously undetected micro-metastasis were observed. The study highlights the generalization challenge even when using a multicenter trained model, and that a small change in indication can considerably impact the model´s performance. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|