Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection
Semantic role labeling involves assigning semantic roles to sentence arguments, providing rich information for various NLP tasks and applications. Annotated corpora with semantic roles are a critical factor in improving the performance of semantic-based models. Besides, Arabic as a low resourced lan...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10820541/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550793300246528 |
|---|---|
| author | Ferial Senator Abdelaziz Lakhfif Imene Zenbout Hanane Boutouta Chahrazed Mediani |
| author_facet | Ferial Senator Abdelaziz Lakhfif Imene Zenbout Hanane Boutouta Chahrazed Mediani |
| author_sort | Ferial Senator |
| collection | DOAJ |
| description | Semantic role labeling involves assigning semantic roles to sentence arguments, providing rich information for various NLP tasks and applications. Annotated corpora with semantic roles are a critical factor in improving the performance of semantic-based models. Besides, Arabic as a low resourced language, have to pay more attention to alternative methods to build such annotated corpora. To this end, two traditional methods have been intensively used, namely, manual annotation and crowed-resourced annotation. The former is highly precise but it demands substantial training and extensive resources, while, the latter, reduce human effort but often results in lower-quality annotations. Recently, Large language model (LLM) based conversational systems like ChatGPT have emerged as a promising tool for text annotation across various NLP tasks. In this paper, we leverage ChatGPT for two main sub-tasks in Arabic language processing. <xref ref-type="disp-formula" rid="deqn1">(1)</xref> Creating an Arabic annotated resource with emotional semantic roles from an English corpus, using cross-lingual annotation projection approach. <xref ref-type="disp-formula" rid="deqn2">(2)</xref> Annotating the Arabic corpus of emotional sentences with emotion categories and semantic roles. Furthermore, we evaluate ChatGPT’s potential for translating English sentences into Arabic. From the perspective of generalization, we test the performance of open-LLMs, specifically, mBERT, and mBART for the same tasks. The evaluation process includes assessing the impact of sentence complexity on the performance of ChatGPT, and open-LLMs in semantic role labeling, and cross-lingual annotation projection. We compared the obtained zero-shot annotation accuracy with that of human base annotations, where the GPT results achieved an accuracy of 0.94 for cross-lingual projection and 0.76 in semantic role labelling, While the open-LLMs achieved notable accuracies of 0.72, and 0.38 respectively. |
| format | Article |
| id | doaj-art-b5c84c2499e84dcca79c236c0b363d1e |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-b5c84c2499e84dcca79c236c0b363d1e2025-01-10T00:01:47ZengIEEEIEEE Access2169-35362025-01-01133707372510.1109/ACCESS.2025.352549310820541Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation ProjectionFerial Senator0https://orcid.org/0009-0004-3024-779XAbdelaziz Lakhfif1Imene Zenbout2https://orcid.org/0000-0002-5168-2449Hanane Boutouta3https://orcid.org/0009-0001-5255-0851Chahrazed Mediani4https://orcid.org/0009-0004-7222-393XDepartment of Computer Science, Faculty of Sciences, Setif 1 University-Ferhat ABBAS, Setif, AlgeriaDepartment of Computer Science, Faculty of Sciences, Setif 1 University-Ferhat ABBAS, Setif, AlgeriaDepartment of Computer Science, Faculty of Sciences, Setif 1 University-Ferhat ABBAS, Setif, AlgeriaDepartment of Computer Science, Faculty of Sciences, Setif 1 University-Ferhat ABBAS, Setif, AlgeriaDepartment of Computer Science, Faculty of Sciences, Setif 1 University-Ferhat ABBAS, Setif, AlgeriaSemantic role labeling involves assigning semantic roles to sentence arguments, providing rich information for various NLP tasks and applications. Annotated corpora with semantic roles are a critical factor in improving the performance of semantic-based models. Besides, Arabic as a low resourced language, have to pay more attention to alternative methods to build such annotated corpora. To this end, two traditional methods have been intensively used, namely, manual annotation and crowed-resourced annotation. The former is highly precise but it demands substantial training and extensive resources, while, the latter, reduce human effort but often results in lower-quality annotations. Recently, Large language model (LLM) based conversational systems like ChatGPT have emerged as a promising tool for text annotation across various NLP tasks. In this paper, we leverage ChatGPT for two main sub-tasks in Arabic language processing. <xref ref-type="disp-formula" rid="deqn1">(1)</xref> Creating an Arabic annotated resource with emotional semantic roles from an English corpus, using cross-lingual annotation projection approach. <xref ref-type="disp-formula" rid="deqn2">(2)</xref> Annotating the Arabic corpus of emotional sentences with emotion categories and semantic roles. Furthermore, we evaluate ChatGPT’s potential for translating English sentences into Arabic. From the perspective of generalization, we test the performance of open-LLMs, specifically, mBERT, and mBART for the same tasks. The evaluation process includes assessing the impact of sentence complexity on the performance of ChatGPT, and open-LLMs in semantic role labeling, and cross-lingual annotation projection. We compared the obtained zero-shot annotation accuracy with that of human base annotations, where the GPT results achieved an accuracy of 0.94 for cross-lingual projection and 0.76 in semantic role labelling, While the open-LLMs achieved notable accuracies of 0.72, and 0.38 respectively.https://ieeexplore.ieee.org/document/10820541/Semantic role labelingcross-lingual annotation projectionemotion analysisChatGPTArabic language |
| spellingShingle | Ferial Senator Abdelaziz Lakhfif Imene Zenbout Hanane Boutouta Chahrazed Mediani Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection IEEE Access Semantic role labeling cross-lingual annotation projection emotion analysis ChatGPT Arabic language |
| title | Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection |
| title_full | Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection |
| title_fullStr | Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection |
| title_full_unstemmed | Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection |
| title_short | Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection |
| title_sort | leveraging chatgpt for enhancing arabic nlp application for semantic role labeling and cross lingual annotation projection |
| topic | Semantic role labeling cross-lingual annotation projection emotion analysis ChatGPT Arabic language |
| url | https://ieeexplore.ieee.org/document/10820541/ |
| work_keys_str_mv | AT ferialsenator leveragingchatgptforenhancingarabicnlpapplicationforsemanticrolelabelingandcrosslingualannotationprojection AT abdelazizlakhfif leveragingchatgptforenhancingarabicnlpapplicationforsemanticrolelabelingandcrosslingualannotationprojection AT imenezenbout leveragingchatgptforenhancingarabicnlpapplicationforsemanticrolelabelingandcrosslingualannotationprojection AT hananeboutouta leveragingchatgptforenhancingarabicnlpapplicationforsemanticrolelabelingandcrosslingualannotationprojection AT chahrazedmediani leveragingchatgptforenhancingarabicnlpapplicationforsemanticrolelabelingandcrosslingualannotationprojection |