Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting
This study explores an innovative approach to early Parkinson’s disease (PD) detection by analyzing speech data collected using a mixed reality (MR) system. A total of 57 Polish participants, including PD patients and healthy controls, performed five speech tasks while using an MR head-mounted displ...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/8/2405 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850179986119458816 |
|---|---|
| author | Milosz Dudek Daria Hemmerling Marta Kaczmarska Joanna Stepien Mateusz Daniol Marek Wodzinski Magdalena Wojcik-Pedziwiatr |
| author_facet | Milosz Dudek Daria Hemmerling Marta Kaczmarska Joanna Stepien Mateusz Daniol Marek Wodzinski Magdalena Wojcik-Pedziwiatr |
| author_sort | Milosz Dudek |
| collection | DOAJ |
| description | This study explores an innovative approach to early Parkinson’s disease (PD) detection by analyzing speech data collected using a mixed reality (MR) system. A total of 57 Polish participants, including PD patients and healthy controls, performed five speech tasks while using an MR head-mounted display (HMD). Speech data were recorded and analyzed to extract acoustic and linguistic features, which were then evaluated using machine learning models, including logistic regression, support vector machines (SVMs), random forests, AdaBoost, and XGBoost. The XGBoost model achieved the best performance, with an F1-score of 0.90 ± 0.05 in the story-retelling task. Key features such as MFCCs (mel-frequency cepstral coefficients), spectral characteristics, RASTA-filtered auditory spectrum, and local shimmer were identified as significant in detecting PD-related speech alterations. Additionally, state-of-the-art deep learning models (wav2vec2, HuBERT, and WavLM) were fine-tuned for PD detection. HuBERT achieved the highest performance, with an F1-score of 0.94 ± 0.04 in the diadochokinetic task, demonstrating the potential of deep learning to capture complex speech patterns linked to neurodegenerative diseases. This study highlights the effectiveness of combining MR technology for speech data collection with advanced machine learning (ML) and deep learning (DL) techniques, offering a non-invasive and high-precision approach to PD diagnosis. The findings hold promise for broader clinical applications, advancing the diagnostic landscape for neurodegenerative disorders. |
| format | Article |
| id | doaj-art-beef3caf53f3486fa5c6dd14d7f116d9 |
| institution | OA Journals |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-beef3caf53f3486fa5c6dd14d7f116d92025-08-20T02:18:20ZengMDPI AGSensors1424-82202025-04-01258240510.3390/s25082405Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality SettingMilosz Dudek0Daria Hemmerling1Marta Kaczmarska2Joanna Stepien3Mateusz Daniol4Marek Wodzinski5Magdalena Wojcik-Pedziwiatr6Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, PolandDepartment of Neurology, Andrzej Frycz Modrzewski Krakow University, 30-705 Krakow, PolandThis study explores an innovative approach to early Parkinson’s disease (PD) detection by analyzing speech data collected using a mixed reality (MR) system. A total of 57 Polish participants, including PD patients and healthy controls, performed five speech tasks while using an MR head-mounted display (HMD). Speech data were recorded and analyzed to extract acoustic and linguistic features, which were then evaluated using machine learning models, including logistic regression, support vector machines (SVMs), random forests, AdaBoost, and XGBoost. The XGBoost model achieved the best performance, with an F1-score of 0.90 ± 0.05 in the story-retelling task. Key features such as MFCCs (mel-frequency cepstral coefficients), spectral characteristics, RASTA-filtered auditory spectrum, and local shimmer were identified as significant in detecting PD-related speech alterations. Additionally, state-of-the-art deep learning models (wav2vec2, HuBERT, and WavLM) were fine-tuned for PD detection. HuBERT achieved the highest performance, with an F1-score of 0.94 ± 0.04 in the diadochokinetic task, demonstrating the potential of deep learning to capture complex speech patterns linked to neurodegenerative diseases. This study highlights the effectiveness of combining MR technology for speech data collection with advanced machine learning (ML) and deep learning (DL) techniques, offering a non-invasive and high-precision approach to PD diagnosis. The findings hold promise for broader clinical applications, advancing the diagnostic landscape for neurodegenerative disorders.https://www.mdpi.com/1424-8220/25/8/2405explainable artificial intelligencelarge language modelsmixed realityParkinson’s diseasevoice biomarkersremote patient monitoring |
| spellingShingle | Milosz Dudek Daria Hemmerling Marta Kaczmarska Joanna Stepien Mateusz Daniol Marek Wodzinski Magdalena Wojcik-Pedziwiatr Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting Sensors explainable artificial intelligence large language models mixed reality Parkinson’s disease voice biomarkers remote patient monitoring |
| title | Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting |
| title_full | Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting |
| title_fullStr | Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting |
| title_full_unstemmed | Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting |
| title_short | Analysis of Voice, Speech, and Language Biomarkers of Parkinson’s Disease Collected in a Mixed Reality Setting |
| title_sort | analysis of voice speech and language biomarkers of parkinson s disease collected in a mixed reality setting |
| topic | explainable artificial intelligence large language models mixed reality Parkinson’s disease voice biomarkers remote patient monitoring |
| url | https://www.mdpi.com/1424-8220/25/8/2405 |
| work_keys_str_mv | AT miloszdudek analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT dariahemmerling analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT martakaczmarska analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT joannastepien analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT mateuszdaniol analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT marekwodzinski analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting AT magdalenawojcikpedziwiatr analysisofvoicespeechandlanguagebiomarkersofparkinsonsdiseasecollectedinamixedrealitysetting |