Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation
<b>Background/Objectives</b>: Accurate assessment of speech intelligibility is necessary for individuals with motor speech disorders. Transcription or scaled rating methods by naïve listeners are the most reliable tasks for these purposes; however, they are often resource-intensive and t...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Diagnostics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-4418/15/15/1892 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849239865328140288 |
|---|---|
| author | Micalle Carl Michal Icht |
| author_facet | Micalle Carl Michal Icht |
| author_sort | Micalle Carl |
| collection | DOAJ |
| description | <b>Background/Objectives</b>: Accurate assessment of speech intelligibility is necessary for individuals with motor speech disorders. Transcription or scaled rating methods by naïve listeners are the most reliable tasks for these purposes; however, they are often resource-intensive and time-consuming within clinical contexts. Automatic speech recognition (ASR) systems, which transcribe speech into text, have been increasingly utilized for assessing speech intelligibility. This study investigates the feasibility of using an open-source ASR system to assess speech intelligibility in Hebrew and English speakers with Down syndrome (DS). <b>Methods</b>: Recordings from 65 Hebrew- and English-speaking participants were included: 33 speakers with DS and 32 typically developing (TD) peers. Speech samples (words, sentences) were transcribed using Whisper (OpenAI) and by naïve listeners. The proportion of agreement between ASR transcriptions and those of naïve listeners was compared across speaker groups (TD, DS) and languages (Hebrew, English) for word-level data. Further comparisons for Hebrew speakers were conducted across speaker groups and stimuli (words, sentences). <b>Results</b>: The strength of the correlation between listener and ASR transcription scores varied across languages, and was higher for English (<i>r</i> = 0.98) than for Hebrew (<i>r</i> = 0.81) for speakers with DS. A higher proportion of listener–ASR agreement was demonstrated for TD speakers, as compared to those with DS (0.94 vs. 0.74, respectively), and for English, in comparison to Hebrew speakers (0.91 for English DS speakers vs. 0.74 for Hebrew DS speakers). Listener–ASR agreement for single words was consistently higher than for sentences among Hebrew speakers. Speakers’ intelligibility influenced word-level agreement among Hebrew- but not English-speaking participants with DS. <b>Conclusions</b>: ASR performance for English closely approximated that of naïve listeners, suggesting potential near-future clinical applicability within single-word intelligibility assessment. In contrast, a lower proportion of agreement between human listeners and ASR for Hebrew speech indicates that broader clinical implementation may require further training of ASR models in this language. |
| format | Article |
| id | doaj-art-3087c3cb905844df97cfdba1a8dc5f67 |
| institution | Kabale University |
| issn | 2075-4418 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Diagnostics |
| spelling | doaj-art-3087c3cb905844df97cfdba1a8dc5f672025-08-20T04:00:49ZengMDPI AGDiagnostics2075-44182025-07-011515189210.3390/diagnostics15151892Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic InvestigationMicalle Carl0Michal Icht1Department of Communication Disorders, Ariel University, Ariel 40700, IsraelDepartment of Communication Disorders, Ariel University, Ariel 40700, Israel<b>Background/Objectives</b>: Accurate assessment of speech intelligibility is necessary for individuals with motor speech disorders. Transcription or scaled rating methods by naïve listeners are the most reliable tasks for these purposes; however, they are often resource-intensive and time-consuming within clinical contexts. Automatic speech recognition (ASR) systems, which transcribe speech into text, have been increasingly utilized for assessing speech intelligibility. This study investigates the feasibility of using an open-source ASR system to assess speech intelligibility in Hebrew and English speakers with Down syndrome (DS). <b>Methods</b>: Recordings from 65 Hebrew- and English-speaking participants were included: 33 speakers with DS and 32 typically developing (TD) peers. Speech samples (words, sentences) were transcribed using Whisper (OpenAI) and by naïve listeners. The proportion of agreement between ASR transcriptions and those of naïve listeners was compared across speaker groups (TD, DS) and languages (Hebrew, English) for word-level data. Further comparisons for Hebrew speakers were conducted across speaker groups and stimuli (words, sentences). <b>Results</b>: The strength of the correlation between listener and ASR transcription scores varied across languages, and was higher for English (<i>r</i> = 0.98) than for Hebrew (<i>r</i> = 0.81) for speakers with DS. A higher proportion of listener–ASR agreement was demonstrated for TD speakers, as compared to those with DS (0.94 vs. 0.74, respectively), and for English, in comparison to Hebrew speakers (0.91 for English DS speakers vs. 0.74 for Hebrew DS speakers). Listener–ASR agreement for single words was consistently higher than for sentences among Hebrew speakers. Speakers’ intelligibility influenced word-level agreement among Hebrew- but not English-speaking participants with DS. <b>Conclusions</b>: ASR performance for English closely approximated that of naïve listeners, suggesting potential near-future clinical applicability within single-word intelligibility assessment. In contrast, a lower proportion of agreement between human listeners and ASR for Hebrew speech indicates that broader clinical implementation may require further training of ASR models in this language.https://www.mdpi.com/2075-4418/15/15/1892speech intelligibilityautomatic speech recognition (ASR)down syndromemotor speech disorderscross-linguistic assessment |
| spellingShingle | Micalle Carl Michal Icht Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation Diagnostics speech intelligibility automatic speech recognition (ASR) down syndrome motor speech disorders cross-linguistic assessment |
| title | Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation |
| title_full | Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation |
| title_fullStr | Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation |
| title_full_unstemmed | Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation |
| title_short | Automated Assessment of Word- and Sentence-Level Speech Intelligibility in Developmental Motor Speech Disorders: A Cross-Linguistic Investigation |
| title_sort | automated assessment of word and sentence level speech intelligibility in developmental motor speech disorders a cross linguistic investigation |
| topic | speech intelligibility automatic speech recognition (ASR) down syndrome motor speech disorders cross-linguistic assessment |
| url | https://www.mdpi.com/2075-4418/15/15/1892 |
| work_keys_str_mv | AT micallecarl automatedassessmentofwordandsentencelevelspeechintelligibilityindevelopmentalmotorspeechdisordersacrosslinguisticinvestigation AT michalicht automatedassessmentofwordandsentencelevelspeechintelligibilityindevelopmentalmotorspeechdisordersacrosslinguisticinvestigation |