Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm
A sentiment analysis is a Natural Language Processing (NLP) task that identifies the opinion or emotional tone of documents such as customer reviews, either at the general or detailed level. Improving domain-specific models is important, as it provides smaller and better-suited models that can be im...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/11/6336 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849721934508457984 |
|---|---|
| author | Daniel Parada Alexandre Branco Marcos Silva Fábio Mendonça Sheikh Mostafa Fernando Morgado-Dias |
| author_facet | Daniel Parada Alexandre Branco Marcos Silva Fábio Mendonça Sheikh Mostafa Fernando Morgado-Dias |
| author_sort | Daniel Parada |
| collection | DOAJ |
| description | A sentiment analysis is a Natural Language Processing (NLP) task that identifies the opinion or emotional tone of documents such as customer reviews, either at the general or detailed level. Improving domain-specific models is important, as it provides smaller and better-suited models that can be implemented by entities that own textual data. This paper presents a deep learning model trained on Portuguese restaurant reviews using recurrent and self-attention mechanisms, which have consistently delivered strong results in prior research studies. Designing an effective model involves numerous hyperparameters and architectural choices. To address this complexity, a discrete genetic algorithm was used to find an optimal configuration, selecting the layer types, placement of self-attention, dropout rate, and model dimensions and shape. A key outcome of this study was that the optimization process produced a model that is competitive with a Bidirectional Encoder Representation from Transformers (BERT) model retrained for Portuguese, which was used as the baseline. The proposed model achieved an area under the curve of 92.1% and F1-score of 75.4%, demonstrating that a small, optimized model can compete and even outperform larger state-of-the-art models. Moreover, this work helps address the scarcity of NLP resources for Portuguese, and highlights the potential of customized architectures over generic solutions. |
| format | Article |
| id | doaj-art-13fb33ee249040faa1ea445d1c91aa13 |
| institution | DOAJ |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-13fb33ee249040faa1ea445d1c91aa132025-08-20T03:11:30ZengMDPI AGApplied Sciences2076-34172025-06-011511633610.3390/app15116336Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic AlgorithmDaniel Parada0Alexandre Branco1Marcos Silva2Fábio Mendonça3Sheikh Mostafa4Fernando Morgado-Dias5Faculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, PortugalFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, PortugalFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, PortugalFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, PortugalITI/LARSyS and ARDITI, 9020-105 Funchal, PortugalFaculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, PortugalA sentiment analysis is a Natural Language Processing (NLP) task that identifies the opinion or emotional tone of documents such as customer reviews, either at the general or detailed level. Improving domain-specific models is important, as it provides smaller and better-suited models that can be implemented by entities that own textual data. This paper presents a deep learning model trained on Portuguese restaurant reviews using recurrent and self-attention mechanisms, which have consistently delivered strong results in prior research studies. Designing an effective model involves numerous hyperparameters and architectural choices. To address this complexity, a discrete genetic algorithm was used to find an optimal configuration, selecting the layer types, placement of self-attention, dropout rate, and model dimensions and shape. A key outcome of this study was that the optimization process produced a model that is competitive with a Bidirectional Encoder Representation from Transformers (BERT) model retrained for Portuguese, which was used as the baseline. The proposed model achieved an area under the curve of 92.1% and F1-score of 75.4%, demonstrating that a small, optimized model can compete and even outperform larger state-of-the-art models. Moreover, this work helps address the scarcity of NLP resources for Portuguese, and highlights the potential of customized architectures over generic solutions.https://www.mdpi.com/2076-3417/15/11/6336deep learningsentiment analysisgenetic algorithmPortuguese language |
| spellingShingle | Daniel Parada Alexandre Branco Marcos Silva Fábio Mendonça Sheikh Mostafa Fernando Morgado-Dias Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm Applied Sciences deep learning sentiment analysis genetic algorithm Portuguese language |
| title | Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm |
| title_full | Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm |
| title_fullStr | Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm |
| title_full_unstemmed | Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm |
| title_short | Optimizing an LSTM Self-Attention Architecture for Portuguese Sentiment Analysis Using a Genetic Algorithm |
| title_sort | optimizing an lstm self attention architecture for portuguese sentiment analysis using a genetic algorithm |
| topic | deep learning sentiment analysis genetic algorithm Portuguese language |
| url | https://www.mdpi.com/2076-3417/15/11/6336 |
| work_keys_str_mv | AT danielparada optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm AT alexandrebranco optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm AT marcossilva optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm AT fabiomendonca optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm AT sheikhmostafa optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm AT fernandomorgadodias optimizinganlstmselfattentionarchitectureforportuguesesentimentanalysisusingageneticalgorithm |