The use of annotations to explain labels: Comparing results from a human‐rater approach to a deep learning approach.
Background: Deep learning methods, where models do not use explicit features and instead rely on implicit features estimated during model training, suffer from an explainability problem. In text classification, saliency maps that reflect the importance of words in prediction are one approach toward...
| Publicado en: | Journal of Computer Assisted Learning Vol. 39; no. 3; pp. 787 - 804 |
|---|---|
| Autores principales: | , , , , |
| Formato: | research tables/charts Journal Article |
| Publicado: |
Wiley-Blackwell
Jun2023
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=163886550&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 163886550 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 02664909 6M1 jtl: Journal of Computer Assisted Learning issn: 02664909 maglogo: Y pubinfo: dt: Jun2023 vid: 39 iid: 3 pid: 480 pub: Wiley-Blackwell place: Malden, Massachusetts artinfo: ui: 163886550 161733982 163886550 163886550 10.1111/jcal.12784 163886550 ppf: 787 ppct: 17 formats: tig: atl: The use of annotations to explain labels: Comparing results from a human‐rater approach to a deep learning approach. aug: au: Lottridge, Susan Woolf, Sherri Young, Mackenzie Jafari, Amir Ormerod, Chris affil: Cambium Assessment, Machine Learning Division, Cambium Learning, Dallas Texas,, USA sug: subj: Data Curation Methods Deep Learning Utilization User-Computer Interface Interrater Reliability Evaluation Validity Human United States Information Management Deep Learning Measurement Issues and Assessments Artificial Intelligence Models, Theoretical ab: Background: Deep learning methods, where models do not use explicit features and instead rely on implicit features estimated during model training, suffer from an explainability problem. In text classification, saliency maps that reflect the importance of words in prediction are one approach toward explainability. However, little is known about whether the salient words agree with those identified by humans as important. Objectives: The current study examines in‐line annotations from human annotators and saliency map annotations from a deep learning model (ELECTRA transformer) to understand how well both humans and machines provide evidence for their assigned label. Methods: Data were responses to test items across a mix of United States subjects, states, and grades. Humans were trained to annotate responses to justify a crisis alert label and two model interpretability methods (LIME, Integrated Gradients) were used to obtain engine annotations. Human inter‐annotator agreement and engine agreement with the human annotators were computed and compared. Results and Conclusions: Human annotators agreed with one another at similar rates to those observed in the literature on similar tasks. The annotations derived using the integrated gradients (IG) agreed with human annotators at higher rates than LIME on most metrics; however, both methods underperformed relative to the human annotators. Implications: Saliency map‐based engine annotations show promise as a form of explanation, but do not reach human annotation agreement levels. Future work should examine the appropriate unit for annotation (e.g., word, sentence), other gradient based methods, and approaches for mapping the continuous saliency values to Boolean annotations. Lay Description: What is known about subject matter: Explainability of deep learning models is complicated by their large size, design, and complexity.Explainability methods in deep learning exist but have not been rigorously tested.Saliency maps, as a form of explainability, can be used to generate engine annotations.Human annotations can serve as basis for comparison. What paper adds to knowledge: How well human annotations work in one domain (crisis paper identification).How well two engine annotation methods match human annotations. Implications of study findings: Human annotations have reasonable reliability.Engine annotations have lower reliability than humans.Model‐based engine annotations perform better than model‐agnostic engine annotations. pubtype: Academic Journal doctype: research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|