Adapting BLOOM to a new language: A case study for the Italian
The BLOOM Large Language Model is a cuttingI think that the authors’ way of doing self-training is quite interesting. I suggest writing about self-training in the abstract and generally expose self-training as one of the paper’s contributions.-edge open linguistic model developed to provide computer...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Accademia University Press
2024-07-01
|
| Series: | IJCoL |
| Online Access: | https://journals.openedition.org/ijcol/1345 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The BLOOM Large Language Model is a cuttingI think that the authors’ way of doing self-training is quite interesting. I suggest writing about self-training in the abstract and generally expose self-training as one of the paper’s contributions.-edge open linguistic model developed to provide computers with natural language understanding skills. Despite its remarkable capabilities in understanding natural language by capturing intricate contextual relationships, the BLOOM model exhibits a notable limitation concerning the number of included languages. In fact, Italian is not included among the languages supported by the model, making its use challenging in this context. Within this study, we explore the language adaptation strategy based on continuing training on language-specific data. Moreover, we fine-tune both the BLOOM and the adapted models on several instruction datasets and different downstream classification tasks over EVALITA datasets. It has been observed that language adaptation followed by instruction-based fine-tuning is shown to be effective in correctly addressing a task never seen by the model in a new language learned on language-specific data. |
|---|---|
| ISSN: | 2499-4553 |