Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis
Abstract In the era of global digital communication, understanding user sentiment across multiple languages is a critical challenge with wide-ranging applications in opinion mining, customer feedback analysis, and social media monitoring. This study advances the field of language-independent sentime...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-03559-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849769271707566080 |
|---|---|
| author | Faizad Ullah Safiullah Faizullah Imdad Ullah Khan Turki Alghamdi Toqeer Ali Syed Ahmad B. Alkhodre Muhammad Sohaib Ayub Asim Karim |
| author_facet | Faizad Ullah Safiullah Faizullah Imdad Ullah Khan Turki Alghamdi Toqeer Ali Syed Ahmad B. Alkhodre Muhammad Sohaib Ayub Asim Karim |
| author_sort | Faizad Ullah |
| collection | DOAJ |
| description | Abstract In the era of global digital communication, understanding user sentiment across multiple languages is a critical challenge with wide-ranging applications in opinion mining, customer feedback analysis, and social media monitoring. This study advances the field of language-independent sentiment analysis by leveraging prompt-based fine-tuning with state-of-the-art transformer models. The performance of classical machine learning approaches, hybrid deep learning architectures, and multilingual transformer models is evaluated across eight typologically diverse languages: Arabic, English, French, German, Hindi, Italian, Portuguese, and Spanish. Baseline models are established using traditional machine learning approaches such as Support Vector Machines (SVM) and Logistic Regression, with feature extraction methods like TF-IDF. A hybrid deep learning model is introduced, combining Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs) to capture local and sequential text patterns. Building on these, pre-trained multilingual transformer models, specifically BERT-base-multilingual and XLM-RoBERTa, are fine-tuned for language-independent sentiment classification tasks. The key contribution lies in the implementation of prompt-based fine-tuning strategies for language independent sentiment analysis. Using (1) prefix prompts and (2) cloze-style prompts, a unified framework is established that employs templates designed in one language and evaluates their performance on data from the remaining $$(n-1)$$ ( n - 1 ) languages. Experimental results demonstrate that transformer models, particularly XLM-RoBERTa with prompt-based fine-tuning outperform both classical and deep learning methods. With only 32 training examples per class, prefix prompts produce results comparable to standard fine-tuning, which typically uses 70-80% of the data for training. This highlights the potential of prompt-based learning for scalable, multilingual sentiment analysis in diverse language settings. |
| format | Article |
| id | doaj-art-4f9599b73b3349e0af6c9643e35fbffe |
| institution | DOAJ |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-4f9599b73b3349e0af6c9643e35fbffe2025-08-20T03:03:28ZengNature PortfolioScientific Reports2045-23222025-07-0115111210.1038/s41598-025-03559-7Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysisFaizad Ullah0Safiullah Faizullah1Imdad Ullah Khan2Turki Alghamdi3Toqeer Ali Syed4Ahmad B. Alkhodre5Muhammad Sohaib Ayub6Asim Karim7Department of Computer Science, LUMSFaculty of Computer and Information Systems, Islamic University of MadinahDepartment of Computer Science, LUMSFaculty of Computer and Information Systems, Islamic University of MadinahFaculty of Computer and Information Systems, Islamic University of MadinahFaculty of Computer and Information Systems, Islamic University of MadinahDepartment of Computer Science, LUMSDepartment of Computer Science, LUMSAbstract In the era of global digital communication, understanding user sentiment across multiple languages is a critical challenge with wide-ranging applications in opinion mining, customer feedback analysis, and social media monitoring. This study advances the field of language-independent sentiment analysis by leveraging prompt-based fine-tuning with state-of-the-art transformer models. The performance of classical machine learning approaches, hybrid deep learning architectures, and multilingual transformer models is evaluated across eight typologically diverse languages: Arabic, English, French, German, Hindi, Italian, Portuguese, and Spanish. Baseline models are established using traditional machine learning approaches such as Support Vector Machines (SVM) and Logistic Regression, with feature extraction methods like TF-IDF. A hybrid deep learning model is introduced, combining Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs) to capture local and sequential text patterns. Building on these, pre-trained multilingual transformer models, specifically BERT-base-multilingual and XLM-RoBERTa, are fine-tuned for language-independent sentiment classification tasks. The key contribution lies in the implementation of prompt-based fine-tuning strategies for language independent sentiment analysis. Using (1) prefix prompts and (2) cloze-style prompts, a unified framework is established that employs templates designed in one language and evaluates their performance on data from the remaining $$(n-1)$$ ( n - 1 ) languages. Experimental results demonstrate that transformer models, particularly XLM-RoBERTa with prompt-based fine-tuning outperform both classical and deep learning methods. With only 32 training examples per class, prefix prompts produce results comparable to standard fine-tuning, which typically uses 70-80% of the data for training. This highlights the potential of prompt-based learning for scalable, multilingual sentiment analysis in diverse language settings.https://doi.org/10.1038/s41598-025-03559-7 |
| spellingShingle | Faizad Ullah Safiullah Faizullah Imdad Ullah Khan Turki Alghamdi Toqeer Ali Syed Ahmad B. Alkhodre Muhammad Sohaib Ayub Asim Karim Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis Scientific Reports |
| title | Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis |
| title_full | Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis |
| title_fullStr | Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis |
| title_full_unstemmed | Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis |
| title_short | Prompt-based fine-tuning with multilingual transformers for language-independent sentiment analysis |
| title_sort | prompt based fine tuning with multilingual transformers for language independent sentiment analysis |
| url | https://doi.org/10.1038/s41598-025-03559-7 |
| work_keys_str_mv | AT faizadullah promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT safiullahfaizullah promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT imdadullahkhan promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT turkialghamdi promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT toqeeralisyed promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT ahmadbalkhodre promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT muhammadsohaibayub promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis AT asimkarim promptbasedfinetuningwithmultilingualtransformersforlanguageindependentsentimentanalysis |