BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews
Online communication has led to more people expressing themselves in their preferred languages, especially in e-commerce, where product reviews are crucial. Understanding customer sentiment through product reviews and comments can help businesses improve product quality and make informed decisions....
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10798428/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850123019619401728 |
|---|---|
| author | Atia Shahnaz Ipa Priyo Nath Roy Mohammad Abu Tareq Rony Ali Raza Norma Latif Fitriyani Yeonghyeon Gu Muhammad Syafrudin |
| author_facet | Atia Shahnaz Ipa Priyo Nath Roy Mohammad Abu Tareq Rony Ali Raza Norma Latif Fitriyani Yeonghyeon Gu Muhammad Syafrudin |
| author_sort | Atia Shahnaz Ipa |
| collection | DOAJ |
| description | Online communication has led to more people expressing themselves in their preferred languages, especially in e-commerce, where product reviews are crucial. Understanding customer sentiment through product reviews and comments can help businesses improve product quality and make informed decisions. However, the complexity of written language and the variety of languages used in reviews pose challenges for accurate sentiment analysis. In this study, we explored the linguistic landscape of Bangladeshi product reviews and developed BdSentiLLM, a robust model designed for automatic language classification and sentiment analysis in this context. We collected a dataset of 3,864 product reviews, revealing that 84% were written in English, followed by Bangla, Banglish (Romanized Bangla), and Bangla-English code-switched content. BdSentiLLM can categorize and prepare these language types for sentiment analysis with large language models. We evaluated the performance of four open-source LLMs, Llama-2, Flan-t5, Vicuna, and Falcon, using BdSentiLLM for sentiment analysis.BdSentiLLM with Llama-2 consistently outperformed the other models across most language categories with f1 score of 0.79 for Bangla, 0.70 for Banglish, 0.84 for Bangla_English, 0.90 for English, and 0.89 overall, while Flan-t5 excelled in English sentiment analysis. Compared to existing models, BdSentiLLM demonstrated superior versatility and effectiveness by handling mixed-language data across all categories making it a valuable tool for sentiment analysis in diverse linguistic contexts. Future work will focus on expanding the dataset to enhance BdSentiLLM’s robustness and exploring its applicability beyond e-commerce to broader multilingual sentiment analysis tasks. |
| format | Article |
| id | doaj-art-40a2323b372e4db895caf6d3a3f5c440 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-40a2323b372e4db895caf6d3a3f5c4402025-08-20T02:34:42ZengIEEEIEEE Access2169-35362024-01-011218933018934310.1109/ACCESS.2024.351682610798428BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product ReviewsAtia Shahnaz Ipa0https://orcid.org/0009-0006-7308-899XPriyo Nath Roy1Mohammad Abu Tareq Rony2https://orcid.org/0000-0002-0640-1425Ali Raza3https://orcid.org/0000-0001-5429-9835Norma Latif Fitriyani4https://orcid.org/0000-0002-1133-3965Yeonghyeon Gu5https://orcid.org/0000-0002-0002-9386Muhammad Syafrudin6https://orcid.org/0000-0002-5640-4413Department of Mechatronics Engineering, Khulna University of Engineering & Technology, Khulna, BangladeshDepartment of Mechatronics Engineering, Khulna University of Engineering & Technology, Khulna, BangladeshDepartment of Statistics, Noakhali Science & Technology University, Noakhali, BangladeshDepartment of Software Engineering, University of Lahore, Lahore, PakistanDepartment of Artificial Intelligence and Data Science, Sejong University, Seoul, Republic of KoreaDepartment of Artificial Intelligence and Data Science, Sejong University, Seoul, Republic of KoreaDepartment of Artificial Intelligence and Data Science, Sejong University, Seoul, Republic of KoreaOnline communication has led to more people expressing themselves in their preferred languages, especially in e-commerce, where product reviews are crucial. Understanding customer sentiment through product reviews and comments can help businesses improve product quality and make informed decisions. However, the complexity of written language and the variety of languages used in reviews pose challenges for accurate sentiment analysis. In this study, we explored the linguistic landscape of Bangladeshi product reviews and developed BdSentiLLM, a robust model designed for automatic language classification and sentiment analysis in this context. We collected a dataset of 3,864 product reviews, revealing that 84% were written in English, followed by Bangla, Banglish (Romanized Bangla), and Bangla-English code-switched content. BdSentiLLM can categorize and prepare these language types for sentiment analysis with large language models. We evaluated the performance of four open-source LLMs, Llama-2, Flan-t5, Vicuna, and Falcon, using BdSentiLLM for sentiment analysis.BdSentiLLM with Llama-2 consistently outperformed the other models across most language categories with f1 score of 0.79 for Bangla, 0.70 for Banglish, 0.84 for Bangla_English, 0.90 for English, and 0.89 overall, while Flan-t5 excelled in English sentiment analysis. Compared to existing models, BdSentiLLM demonstrated superior versatility and effectiveness by handling mixed-language data across all categories making it a valuable tool for sentiment analysis in diverse linguistic contexts. Future work will focus on expanding the dataset to enhance BdSentiLLM’s robustness and exploring its applicability beyond e-commerce to broader multilingual sentiment analysis tasks.https://ieeexplore.ieee.org/document/10798428/Sentiment analysisproductBangladeshi reviewslarge language modelnatural language processing |
| spellingShingle | Atia Shahnaz Ipa Priyo Nath Roy Mohammad Abu Tareq Rony Ali Raza Norma Latif Fitriyani Yeonghyeon Gu Muhammad Syafrudin BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews IEEE Access Sentiment analysis product Bangladeshi reviews large language model natural language processing |
| title | BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews |
| title_full | BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews |
| title_fullStr | BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews |
| title_full_unstemmed | BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews |
| title_short | BdSentiLLM: A Novel LLM Approach to Sentiment Analysis of Product Reviews |
| title_sort | bdsentillm a novel llm approach to sentiment analysis of product reviews |
| topic | Sentiment analysis product Bangladeshi reviews large language model natural language processing |
| url | https://ieeexplore.ieee.org/document/10798428/ |
| work_keys_str_mv | AT atiashahnazipa bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT priyonathroy bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT mohammadabutareqrony bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT aliraza bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT normalatiffitriyani bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT yeonghyeongu bdsentillmanovelllmapproachtosentimentanalysisofproductreviews AT muhammadsyafrudin bdsentillmanovelllmapproachtosentimentanalysisofproductreviews |