Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities.
Research into clinical applications of speech-based emotion recognition (SER) technologies has been steadily increasing over the past few years. One such potential application is the automatic recognition of expressed emotion (EE) components within family environments. The identification of EE is hi...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2024-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0300518 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841556857377783808 |
|---|---|
| author | Bahman Mirheidari André Bittar Nicholas Cummins Johnny Downs Helen L Fisher Heidi Christensen |
| author_facet | Bahman Mirheidari André Bittar Nicholas Cummins Johnny Downs Helen L Fisher Heidi Christensen |
| author_sort | Bahman Mirheidari |
| collection | DOAJ |
| description | Research into clinical applications of speech-based emotion recognition (SER) technologies has been steadily increasing over the past few years. One such potential application is the automatic recognition of expressed emotion (EE) components within family environments. The identification of EE is highly important as they have been linked with a range of adverse life events. Manual coding of these events requires time-consuming specialist training, amplifying the need for automated approaches. Herein we describe an automated machine learning approach for determining the degree of warmth, a key component of EE, from acoustic and text natural language features. Our dataset of 52 recorded interviews is taken from recordings, collected over 20 years ago, from a nationally representative birth cohort of British twin children, and was manually coded for EE by two researchers (inter-rater reliability 0.84-0.90). We demonstrate that the degree of warmth can be predicted with an F1-score of 64.7% despite working with audio recordings of highly variable quality. Our highly promising results suggest that machine learning may be able to assist in the coding of EE in the near future. |
| format | Article |
| id | doaj-art-4ec243217a3341efa68b20742015ea96 |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-4ec243217a3341efa68b20742015ea962025-01-07T05:33:31ZengPublic Library of Science (PLoS)PLoS ONE1932-62032024-01-01193e030051810.1371/journal.pone.0300518Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities.Bahman MirheidariAndré BittarNicholas CumminsJohnny DownsHelen L FisherHeidi ChristensenResearch into clinical applications of speech-based emotion recognition (SER) technologies has been steadily increasing over the past few years. One such potential application is the automatic recognition of expressed emotion (EE) components within family environments. The identification of EE is highly important as they have been linked with a range of adverse life events. Manual coding of these events requires time-consuming specialist training, amplifying the need for automated approaches. Herein we describe an automated machine learning approach for determining the degree of warmth, a key component of EE, from acoustic and text natural language features. Our dataset of 52 recorded interviews is taken from recordings, collected over 20 years ago, from a nationally representative birth cohort of British twin children, and was manually coded for EE by two researchers (inter-rater reliability 0.84-0.90). We demonstrate that the degree of warmth can be predicted with an F1-score of 64.7% despite working with audio recordings of highly variable quality. Our highly promising results suggest that machine learning may be able to assist in the coding of EE in the near future.https://doi.org/10.1371/journal.pone.0300518 |
| spellingShingle | Bahman Mirheidari André Bittar Nicholas Cummins Johnny Downs Helen L Fisher Heidi Christensen Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. PLoS ONE |
| title | Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. |
| title_full | Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. |
| title_fullStr | Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. |
| title_full_unstemmed | Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. |
| title_short | Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities. |
| title_sort | automatic detection of expressed emotion from five minute speech samples challenges and opportunities |
| url | https://doi.org/10.1371/journal.pone.0300518 |
| work_keys_str_mv | AT bahmanmirheidari automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities AT andrebittar automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities AT nicholascummins automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities AT johnnydowns automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities AT helenlfisher automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities AT heidichristensen automaticdetectionofexpressedemotionfromfiveminutespeechsampleschallengesandopportunities |