Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study
Multiple-choice questions are commonly used to assess knowledge through a set of possible answers to a given question. Determining the correct answer relies on the balance between understanding the question’s content and the associated logic. Generative models are widely applied in Multiple-Choice...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
LibraryPress@UF
2025-05-01
|
| Series: | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| Subjects: | |
| Online Access: | https://journals.flvc.org/FLAIRS/article/view/138969 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849726721948909568 |
|---|---|
| author | Guilherme Dallmann Lima Emerson Lopes Henry Pereira Marilia Silveira Larissa Freitas Ulisses Corrêa |
| author_facet | Guilherme Dallmann Lima Emerson Lopes Henry Pereira Marilia Silveira Larissa Freitas Ulisses Corrêa |
| author_sort | Guilherme Dallmann Lima |
| collection | DOAJ |
| description |
Multiple-choice questions are commonly used to assess knowledge through a set of possible answers to a given question. Determining the correct answer relies on the balance between understanding the question’s content and the associated logic. Generative models are widely applied in Multiple-Choice Question Answering (MCQA) tasks, as they can process the context and predict the correct answer based on the provided input. In this regard, the language used in the question is a critical factor, as comprehension may require understanding linguistic nuances. This work investigates the performance of transformer-based generative models in the MCQA task for Portuguese, under both zero-shot and one-shot scenarios. We compare monolingual (Sabiá-7B and Tucano-2B4) and multilingual (LLaMA-8B and LLaMA-3B) models on MCQA datasets focused on college entrance exams, aiming to evaluate the influence of prior knowledge and the model's adaptation to complex languages. Our results demonstrate that, although LLaMA-8B was not specifically trained for Portuguese, it outperforms the Sabiá-7B model on the ENEM-Challenge and BLUEX datasets. Finally, we show that multilingual models with more recent architectures outperform monolingual models.
|
| format | Article |
| id | doaj-art-e5e145223fb94a9d90dcfc4455bb58d8 |
| institution | DOAJ |
| issn | 2334-0754 2334-0762 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | LibraryPress@UF |
| record_format | Article |
| series | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| spelling | doaj-art-e5e145223fb94a9d90dcfc4455bb58d82025-08-20T03:10:06ZengLibraryPress@UFProceedings of the International Florida Artificial Intelligence Research Society Conference2334-07542334-07622025-05-0138110.32473/flairs.38.1.138969Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental StudyGuilherme Dallmann Lima0Emerson Lopes1Henry Pereira2Marilia Silveira3Larissa Freitas4Ulisses Corrêa5University Federal of PelotasFederal University of PelotasFederal University of PelotasFederal University of PelotasFederal University of PelotasFederal University of Pelotas Multiple-choice questions are commonly used to assess knowledge through a set of possible answers to a given question. Determining the correct answer relies on the balance between understanding the question’s content and the associated logic. Generative models are widely applied in Multiple-Choice Question Answering (MCQA) tasks, as they can process the context and predict the correct answer based on the provided input. In this regard, the language used in the question is a critical factor, as comprehension may require understanding linguistic nuances. This work investigates the performance of transformer-based generative models in the MCQA task for Portuguese, under both zero-shot and one-shot scenarios. We compare monolingual (Sabiá-7B and Tucano-2B4) and multilingual (LLaMA-8B and LLaMA-3B) models on MCQA datasets focused on college entrance exams, aiming to evaluate the influence of prior knowledge and the model's adaptation to complex languages. Our results demonstrate that, although LLaMA-8B was not specifically trained for Portuguese, it outperforms the Sabiá-7B model on the ENEM-Challenge and BLUEX datasets. Finally, we show that multilingual models with more recent architectures outperform monolingual models. https://journals.flvc.org/FLAIRS/article/view/138969TransformersQuestion AnsweringGenerative AI |
| spellingShingle | Guilherme Dallmann Lima Emerson Lopes Henry Pereira Marilia Silveira Larissa Freitas Ulisses Corrêa Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study Proceedings of the International Florida Artificial Intelligence Research Society Conference Transformers Question Answering Generative AI |
| title | Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study |
| title_full | Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study |
| title_fullStr | Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study |
| title_full_unstemmed | Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study |
| title_short | Generative Models for Multiple-Choice Question Answering in Portuguese: A Monolingual and Multilingual Experimental Study |
| title_sort | generative models for multiple choice question answering in portuguese a monolingual and multilingual experimental study |
| topic | Transformers Question Answering Generative AI |
| url | https://journals.flvc.org/FLAIRS/article/view/138969 |
| work_keys_str_mv | AT guilhermedallmannlima generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy AT emersonlopes generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy AT henrypereira generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy AT mariliasilveira generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy AT larissafreitas generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy AT ulissescorrea generativemodelsformultiplechoicequestionansweringinportugueseamonolingualandmultilingualexperimentalstudy |