Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network
Music emotion recognition (MER) seeks to understand the complex emotional landscapes elicited by music, acknowledging music’s profound social and psychological roles beyond traditional tasks such as genre classification or content similarity. MER relies heavily on high‑quality emotional annotations,...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ubiquity Press
2025-06-01
|
| Series: | Transactions of the International Society for Music Information Retrieval |
| Subjects: | |
| Online Access: | https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/235 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849716698750386176 |
|---|---|
| author | Andreas Peintner Marta Moscati Yu Kinoshita Richard Vogl Peter Knees Markus Schedl Hannah Strauss Marcel Zentner Eva Zangerle |
| author_facet | Andreas Peintner Marta Moscati Yu Kinoshita Richard Vogl Peter Knees Markus Schedl Hannah Strauss Marcel Zentner Eva Zangerle |
| author_sort | Andreas Peintner |
| collection | DOAJ |
| description | Music emotion recognition (MER) seeks to understand the complex emotional landscapes elicited by music, acknowledging music’s profound social and psychological roles beyond traditional tasks such as genre classification or content similarity. MER relies heavily on high‑quality emotional annotations, which serve as the foundation for training models to recognize emotions. However, collecting these annotations is both complex and costly, leading to limited availability of large‑scale datasets for MER. Recent efforts in MER for automatically extracting emotion have focused on learning track representations in a supervised manner. However, these approaches mainly use simplified emotion models due to limited datasets or a lack of necessity for sophisticated emotion models and ignore hidden inter‑track relations, which are beneficial in a semi‑supervised learning setting. This paper proposes a novel approach to MER by constructing a multi‑relational graph that encapsulates different facets of music. We leverage graph neural networks to model intricate inter‑track relationships and capture structurally induced representations from user data, such as listening histories, genres, and tags. Our model, the semi‑supervised multi‑relational graph neural network for emotion recognition (SRGNN‑Emo), innovates by combining graph‑based modeling with semi‑supervised learning, using rich user data to extract nuanced emotional profiles from music tracks. Through extensive experimentation, SRGNN‑Emo demonstrates significant improvements in R2 and root mean squared error metrics for predicting the intensity of nine continuous emotions (Geneva Emotional Music Scale), demonstrating its superior capability in capturing and predicting complex emotional expressions in music. |
| format | Article |
| id | doaj-art-a2174c70959c491f910284cb1b4b2d93 |
| institution | DOAJ |
| issn | 2514-3298 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Ubiquity Press |
| record_format | Article |
| series | Transactions of the International Society for Music Information Retrieval |
| spelling | doaj-art-a2174c70959c491f910284cb1b4b2d932025-08-20T03:12:54ZengUbiquity PressTransactions of the International Society for Music Information Retrieval2514-32982025-06-0181140–153140–15310.5334/tismir.235235Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural NetworkAndreas Peintner0https://orcid.org/0000-0001-7337-524XMarta Moscati1https://orcid.org/0000-0002-5541-4919Yu Kinoshita2Richard Vogl3https://orcid.org/0000-0003-2488-0084Peter Knees4https://orcid.org/0000-0003-3906-1292Markus Schedl5https://orcid.org/0000-0003-1706-3406Hannah Strauss6https://orcid.org/0000-0002-6235-4010Marcel Zentner7https://orcid.org/0000-0001-8580-8030Eva Zangerle8https://orcid.org/0000-0003-3195-8273University of Innsbruck, InnsbruckJohannes Kepler University Linz, LinzTU Wien, WienTU Wien, WienTU Wien, WienJohannes Kepler University Linz, LinzUniversity of Innsbruck, InnsbruckUniversity of Innsbruck, InnsbruckUniversity of Innsbruck, InnsbruckMusic emotion recognition (MER) seeks to understand the complex emotional landscapes elicited by music, acknowledging music’s profound social and psychological roles beyond traditional tasks such as genre classification or content similarity. MER relies heavily on high‑quality emotional annotations, which serve as the foundation for training models to recognize emotions. However, collecting these annotations is both complex and costly, leading to limited availability of large‑scale datasets for MER. Recent efforts in MER for automatically extracting emotion have focused on learning track representations in a supervised manner. However, these approaches mainly use simplified emotion models due to limited datasets or a lack of necessity for sophisticated emotion models and ignore hidden inter‑track relations, which are beneficial in a semi‑supervised learning setting. This paper proposes a novel approach to MER by constructing a multi‑relational graph that encapsulates different facets of music. We leverage graph neural networks to model intricate inter‑track relationships and capture structurally induced representations from user data, such as listening histories, genres, and tags. Our model, the semi‑supervised multi‑relational graph neural network for emotion recognition (SRGNN‑Emo), innovates by combining graph‑based modeling with semi‑supervised learning, using rich user data to extract nuanced emotional profiles from music tracks. Through extensive experimentation, SRGNN‑Emo demonstrates significant improvements in R2 and root mean squared error metrics for predicting the intensity of nine continuous emotions (Geneva Emotional Music Scale), demonstrating its superior capability in capturing and predicting complex emotional expressions in music.https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/235music emotion recognitiongraph neural networkssemi-supervised learningcontrastive lossevoked emotionsnode representationmulti-relationalgenrestags |
| spellingShingle | Andreas Peintner Marta Moscati Yu Kinoshita Richard Vogl Peter Knees Markus Schedl Hannah Strauss Marcel Zentner Eva Zangerle Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network Transactions of the International Society for Music Information Retrieval music emotion recognition graph neural networks semi-supervised learning contrastive loss evoked emotions node representation multi-relational genres tags |
| title | Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network |
| title_full | Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network |
| title_fullStr | Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network |
| title_full_unstemmed | Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network |
| title_short | Nuanced Music Emotion Recognition via a Semi‑Supervised Multi‑Relational Graph Neural Network |
| title_sort | nuanced music emotion recognition via a semi supervised multi relational graph neural network |
| topic | music emotion recognition graph neural networks semi-supervised learning contrastive loss evoked emotions node representation multi-relational genres tags |
| url | https://account.transactions.ismir.net/index.php/up-j-tismir/article/view/235 |
| work_keys_str_mv | AT andreaspeintner nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT martamoscati nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT yukinoshita nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT richardvogl nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT peterknees nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT markusschedl nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT hannahstrauss nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT marcelzentner nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork AT evazangerle nuancedmusicemotionrecognitionviaasemisupervisedmultirelationalgraphneuralnetwork |