Validity of Off-the-Shelf Automatic Speech Recognition for Assessing Speech Intelligibility and Speech Severity in Speakers With Amyotrophic Lateral Sclerosis.
Purpose: There is increasing interest in using automatic speech recognition (ASR) systems to evaluate impairment severity or speech intelligibility in speakers with dysarthria. We assessed the clinical validity of one currently available off-the-shelf (OTS) ASR system (i.e., a Google Cloud ASR API)...
| Publicado en: | Journal of Speech, Language & Hearing Research Vol. 65; no. 6; pp. 2128 - 2144 |
|---|---|
| Autores principales: | , , , , |
| Formato: | Artículo |
| Publicado: |
American Speech-Language-Hearing Association
Jun2022
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=157403213&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 157403213 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 10924388 1SM jtl: Journal of Speech, Language & Hearing Research issn: 10924388 maglogo: N pubinfo: dt: Jun2022 vid: 65 iid: 6 pid: 42 pub: American Speech-Language-Hearing Association artinfo: ui: 157403213 10.1044/2022_JSLHR-21-00589 ppf: 2128 ppct: 16 formats: fmt: @attributes: type: P size: 1.3MB tig: atl: Validity of Off-the-Shelf Automatic Speech Recognition for Assessing Speech Intelligibility and Speech Severity in Speakers With Amyotrophic Lateral Sclerosis. aug: au: Gutz, Sarah E. Stipancic, Kaila L. Yunusova, Yana Berry, James D. Green, Jordan R. affil: Program in Speech and Hearing Bioscience and Technology, Harvard Medical School, Boston, MA Department of Communicative Disorders and Sciences, University at Buffalo, NY Department of Speech-Language Pathology, University of Toronto, Ontario, Canada Hurvitz Brain Sciences Program, Sunnybrook Research Institute, Toronto, Ontario, Canada Toronto Rehabilitation Institute, University Health Network, Ontario, Canada Sean M. Healey and AMG Center for ALS, Massachusetts General Hospital, Boston Department of Communication Sciences and Disorders, MGH Institute of Health Professions, Boston, MA su: Judgment (Psychology) Automatic speech recognition Intelligibility of speech Amyotrophic lateral sclerosis Speech disorder diagnosis Speech evaluation Noise Quality assurance sug: subj: Judgment (Psychology) Automatic speech recognition Intelligibility of speech Amyotrophic lateral sclerosis Speech disorder diagnosis Speech evaluation Noise Quality assurance ab: Purpose: There is increasing interest in using automatic speech recognition (ASR) systems to evaluate impairment severity or speech intelligibility in speakers with dysarthria. We assessed the clinical validity of one currently available off-the-shelf (OTS) ASR system (i.e., a Google Cloud ASR API) for indexing sentence-level speech intelligibility and impairment severity in individuals with amyotrophic lateral sclerosis (ALS), and we provided guidance for potential users of such systems in research and clinic. Method: Using speech samples collected from 52 individuals with ALS and 20 healthy control speakers, we compared word recognition rate (WRR) from the commercially available Google Cloud ASR API (Machine WRR) to clinicianprovided judgments of impairment severity, as well as sentence intelligibility (Human WRR). We assessed the internal reliability of Machine and Human WRR by comparing the standard deviation of WRR across sentences to the minimally detectable change (MDC), a clinical benchmark that indicates whether results are within measurement error. We also evaluated Machine and Human WRR diagnostic accuracy for classifying speakers into clinically established categories. Results: Human WRR achieved better accuracy than Machine WRR when indexing speech severity, and, although related, Human and Machine WRR were not strongly correlated. When the speech signal was mixed with noise (noise-augmented ASR) to reduce a ceiling effect, Machine WRR performance improved. Internal reliability metrics were worse for Machine than Human WRR, particularly for typical and mildly impaired severity groups, although sentence length significantly impacted both Machine and Human WRRs. Conclusions: Results indicated that the OTS ASR system was inadequate for early detection of speech impairment and grading overall speech severity. While Machine and Human WRR were correlated, ASR should not be used as a one-toone proxy for transcription speech intelligibility or clinician severity ratings. Overall, findings suggested that the tested OTS ASR system, Google Cloud ASR, has limited utility for grading clinical speech impairment in speakers with ALS. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|