Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation
Empathetic and coherent responses are critical in automated chatbot-facilitated psychotherapy. This study addresses the challenge of enhancing the emotional and contextual understanding of large language models (LLMs) in psychiatric applications. We introduce Emotion-Aware Embedding Fusion, a novel...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | AI |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-2688/6/3/56 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850093692644229120 |
|---|---|
| author | Abdur Rasool Muhammad Irfan Shahzad Hafsa Aslam Vincent Chan Muhammad Ali Arshad |
| author_facet | Abdur Rasool Muhammad Irfan Shahzad Hafsa Aslam Vincent Chan Muhammad Ali Arshad |
| author_sort | Abdur Rasool |
| collection | DOAJ |
| description | Empathetic and coherent responses are critical in automated chatbot-facilitated psychotherapy. This study addresses the challenge of enhancing the emotional and contextual understanding of large language models (LLMs) in psychiatric applications. We introduce Emotion-Aware Embedding Fusion, a novel framework integrating hierarchical fusion and attention mechanisms to prioritize semantic and emotional features in therapy transcripts. Our approach combines multiple emotion lexicons, including NRC Emotion Lexicon, VADER, WordNet, and SentiWordNet, with state-of-the-art LLMs such as Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4. Therapy session transcripts, comprising over 2000 samples, are segmented into hierarchical levels (word, sentence, and session) using neural networks, while hierarchical fusion combines these features with pooling techniques to refine emotional representations. Attention mechanisms, including multi-head self-attention and cross-attention, further prioritize emotional and contextual features, enabling the temporal modeling of emotional shifts across sessions. The processed embeddings, computed using BERT, GPT-3, and RoBERTa, are stored in the Facebook AI similarity search vector database, which enables efficient similarity search and clustering across dense vector spaces. Upon user queries, relevant segments are retrieved and provided as context to LLMs, enhancing their ability to generate empathetic and contextually relevant responses. The proposed framework is evaluated across multiple practical use cases to demonstrate real-world applicability, including AI-driven therapy chatbots. The system can be integrated into existing mental health platforms to generate personalized responses based on retrieved therapy session data. The experimental results show that our framework enhances empathy, coherence, informativeness, and fluency, surpassing baseline models while improving LLMs’ emotional intelligence and contextual adaptability for psychotherapy. |
| format | Article |
| id | doaj-art-7bdddcc5de654a6797a3a639f5858062 |
| institution | DOAJ |
| issn | 2673-2688 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | AI |
| spelling | doaj-art-7bdddcc5de654a6797a3a639f58580622025-08-20T02:41:51ZengMDPI AGAI2673-26882025-03-01635610.3390/ai6030056Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response GenerationAbdur Rasool0Muhammad Irfan Shahzad1Hafsa Aslam2Vincent Chan3Muhammad Ali Arshad4Department of Information and Computer Sciences, University of Hawaii at Manoa, Honolulu, HI 96822, USASelTeq, University Avenue, Palo Alto, CA 94301, USASchool of Computer Science and Technology, Donghua University, Shanghai 201620, ChinaDepartment of Information and Computer Sciences, University of Hawaii at Manoa, Honolulu, HI 96822, USADepartment of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, ChinaEmpathetic and coherent responses are critical in automated chatbot-facilitated psychotherapy. This study addresses the challenge of enhancing the emotional and contextual understanding of large language models (LLMs) in psychiatric applications. We introduce Emotion-Aware Embedding Fusion, a novel framework integrating hierarchical fusion and attention mechanisms to prioritize semantic and emotional features in therapy transcripts. Our approach combines multiple emotion lexicons, including NRC Emotion Lexicon, VADER, WordNet, and SentiWordNet, with state-of-the-art LLMs such as Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4. Therapy session transcripts, comprising over 2000 samples, are segmented into hierarchical levels (word, sentence, and session) using neural networks, while hierarchical fusion combines these features with pooling techniques to refine emotional representations. Attention mechanisms, including multi-head self-attention and cross-attention, further prioritize emotional and contextual features, enabling the temporal modeling of emotional shifts across sessions. The processed embeddings, computed using BERT, GPT-3, and RoBERTa, are stored in the Facebook AI similarity search vector database, which enables efficient similarity search and clustering across dense vector spaces. Upon user queries, relevant segments are retrieved and provided as context to LLMs, enhancing their ability to generate empathetic and contextually relevant responses. The proposed framework is evaluated across multiple practical use cases to demonstrate real-world applicability, including AI-driven therapy chatbots. The system can be integrated into existing mental health platforms to generate personalized responses based on retrieved therapy session data. The experimental results show that our framework enhances empathy, coherence, informativeness, and fluency, surpassing baseline models while improving LLMs’ emotional intelligence and contextual adaptability for psychotherapy.https://www.mdpi.com/2673-2688/6/3/56large language modelspsychotherapy chatbotsemotion-aware embeddinghierarchical fusionattention mechanismsemotion lexicon |
| spellingShingle | Abdur Rasool Muhammad Irfan Shahzad Hafsa Aslam Vincent Chan Muhammad Ali Arshad Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation AI large language models psychotherapy chatbots emotion-aware embedding hierarchical fusion attention mechanisms emotion lexicon |
| title | Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation |
| title_full | Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation |
| title_fullStr | Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation |
| title_full_unstemmed | Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation |
| title_short | Emotion-Aware Embedding Fusion in Large Language Models (Flan-T5, Llama 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation |
| title_sort | emotion aware embedding fusion in large language models flan t5 llama 2 deepseek r1 and chatgpt 4 for intelligent response generation |
| topic | large language models psychotherapy chatbots emotion-aware embedding hierarchical fusion attention mechanisms emotion lexicon |
| url | https://www.mdpi.com/2673-2688/6/3/56 |
| work_keys_str_mv | AT abdurrasool emotionawareembeddingfusioninlargelanguagemodelsflant5llama2deepseekr1andchatgpt4forintelligentresponsegeneration AT muhammadirfanshahzad emotionawareembeddingfusioninlargelanguagemodelsflant5llama2deepseekr1andchatgpt4forintelligentresponsegeneration AT hafsaaslam emotionawareembeddingfusioninlargelanguagemodelsflant5llama2deepseekr1andchatgpt4forintelligentresponsegeneration AT vincentchan emotionawareembeddingfusioninlargelanguagemodelsflant5llama2deepseekr1andchatgpt4forintelligentresponsegeneration AT muhammadaliarshad emotionawareembeddingfusioninlargelanguagemodelsflant5llama2deepseekr1andchatgpt4forintelligentresponsegeneration |