U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models
This paper investigates the rapidly advancing domain of Large Language Models (LLMs) and their growing potential in various fields. A central focus is the exploration of LLMs, e.g., LLaMA, as powerful tools for modeling and representing linguistic information, especially in the realm of syntax. We a...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Accademia University Press
2024-07-01
|
| Series: | IJCoL |
| Online Access: | https://journals.openedition.org/ijcol/1352 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850124186538737664 |
|---|---|
| author | Claudiu Daniel Hromei Danilo Croce Roberto Basili |
| author_facet | Claudiu Daniel Hromei Danilo Croce Roberto Basili |
| author_sort | Claudiu Daniel Hromei |
| collection | DOAJ |
| description | This paper investigates the rapidly advancing domain of Large Language Models (LLMs) and their growing potential in various fields. A central focus is the exploration of LLMs, e.g., LLaMA, as powerful tools for modeling and representing linguistic information, especially in the realm of syntax. We aim to evaluate the ability of these models to encode syntactic information, especially when explicitly supplied, through fine-tuning processes. Traditionally, Dependency Parsing has relied on specific techniques and dedicated architectures. Our research shifts this approach, conceptualizing it as a sequence-to-sequence task where Language Models interpret and transform syntax into bracketed structures that reflect dependency graphs. We introduce U-DepPLLaMA (Universal Dependency Parsing via auto-regressive LLMs based on LLaMA), a novel architecture optimized for multilingual, end-to-end Dependency Parsing. Our experimental evaluation, across 50 datasets in 26 languages from the Universal Dependency Treebank, shows that LLMs can be effectively trained for dependency parsing without the need for task-specific architectures. The results are on par with current state-of-the-art methods and demonstrate resilience across varying sentence complexities and lengths. |
| format | Article |
| id | doaj-art-332e8ece7ea747c1ac0cfedbb3084324 |
| institution | OA Journals |
| issn | 2499-4553 |
| language | English |
| publishDate | 2024-07-01 |
| publisher | Accademia University Press |
| record_format | Article |
| series | IJCoL |
| spelling | doaj-art-332e8ece7ea747c1ac0cfedbb30843242025-08-20T02:34:24ZengAccademia University PressIJCoL2499-45532024-07-0110110.4000/125nmU-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language ModelsClaudiu Daniel HromeiDanilo CroceRoberto BasiliThis paper investigates the rapidly advancing domain of Large Language Models (LLMs) and their growing potential in various fields. A central focus is the exploration of LLMs, e.g., LLaMA, as powerful tools for modeling and representing linguistic information, especially in the realm of syntax. We aim to evaluate the ability of these models to encode syntactic information, especially when explicitly supplied, through fine-tuning processes. Traditionally, Dependency Parsing has relied on specific techniques and dedicated architectures. Our research shifts this approach, conceptualizing it as a sequence-to-sequence task where Language Models interpret and transform syntax into bracketed structures that reflect dependency graphs. We introduce U-DepPLLaMA (Universal Dependency Parsing via auto-regressive LLMs based on LLaMA), a novel architecture optimized for multilingual, end-to-end Dependency Parsing. Our experimental evaluation, across 50 datasets in 26 languages from the Universal Dependency Treebank, shows that LLMs can be effectively trained for dependency parsing without the need for task-specific architectures. The results are on par with current state-of-the-art methods and demonstrate resilience across varying sentence complexities and lengths.https://journals.openedition.org/ijcol/1352 |
| spellingShingle | Claudiu Daniel Hromei Danilo Croce Roberto Basili U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models IJCoL |
| title | U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models |
| title_full | U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models |
| title_fullStr | U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models |
| title_full_unstemmed | U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models |
| title_short | U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models |
| title_sort | u deppllama universal dependency parsing via auto regressive large language models |
| url | https://journals.openedition.org/ijcol/1352 |
| work_keys_str_mv | AT claudiudanielhromei udeppllamauniversaldependencyparsingviaautoregressivelargelanguagemodels AT danilocroce udeppllamauniversaldependencyparsingviaautoregressivelargelanguagemodels AT robertobasili udeppllamauniversaldependencyparsingviaautoregressivelargelanguagemodels |