AfriSign: African sign languages machine translation
Abstract Research on sign language translation is ongoing with a high social inclusive goal of crossing the bridge between people with hearing disability using sign language as their basic way to communicate to others who do not understand sign language. Hundreds of different sign languages exist in...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-01-01
|
| Series: | Discover Artificial Intelligence |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44163-025-00227-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585546320314368 |
|---|---|
| author | Kate Takyi Rose-Mary Owusuaa Mensah Gyening Shester Msouobu Gueuwou Marco Stanley Nyarko Richard Adade Reindorf Nartey Borkor Samuelson Israel Boadu-Acheampong Linus Tabari |
| author_facet | Kate Takyi Rose-Mary Owusuaa Mensah Gyening Shester Msouobu Gueuwou Marco Stanley Nyarko Richard Adade Reindorf Nartey Borkor Samuelson Israel Boadu-Acheampong Linus Tabari |
| author_sort | Kate Takyi |
| collection | DOAJ |
| description | Abstract Research on sign language translation is ongoing with a high social inclusive goal of crossing the bridge between people with hearing disability using sign language as their basic way to communicate to others who do not understand sign language. Hundreds of different sign languages exist instead of a single universal sign language. Research on translating sign languages from high-income nations has grown significantly, but little is known about translating sign languages from Africa. In this paper, we curate a novel video-to-text African sign languages translation dataset containing sign language videos of Bible verses from six (6) different African countries. We experimented with competitive machine translation and sign language translation techniques on our dataset, including the application of transformers to sign language translation, multilingual training, and cross-transfer learning. We evaluated them in terms of accuracy and precision. The results from our experiments prove that having one Multilingual model for all the languages tends to be a better choice when deployed in real system in terms of memory usage with an accuracy of 94.6% and precision of 97.3%. These results give headway for more multilingual models to be developed to enhance inclusion for the deaf community and bridge the gap between the hearing and the deaf in Africa. |
| format | Article |
| id | doaj-art-1227439974314eb0a8cb835035127254 |
| institution | Kabale University |
| issn | 2731-0809 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Springer |
| record_format | Article |
| series | Discover Artificial Intelligence |
| spelling | doaj-art-1227439974314eb0a8cb8350351272542025-01-26T12:43:01ZengSpringerDiscover Artificial Intelligence2731-08092025-01-015111610.1007/s44163-025-00227-7AfriSign: African sign languages machine translationKate Takyi0Rose-Mary Owusuaa Mensah Gyening1Shester Msouobu Gueuwou2Marco Stanley Nyarko3Richard Adade4Reindorf Nartey Borkor5Samuelson Israel Boadu-Acheampong6Linus Tabari7Department of Computer Science, Kwame Nkrumah University of Science and TechnologyDepartment of Computer Science, Kwame Nkrumah University of Science and TechnologyDepartment of Computer Science, Kwame Nkrumah University of Science and TechnologyDepartment of Disability and Rehabilitation Studies, Kwame Nkrumah University of Science and TechnologyDepartment of Disability and Rehabilitation Studies, Kwame Nkrumah University of Science and TechnologyDepartment of Mathematics, Kwame Nkrumah University of Science and TechnologyDepartment of Computer Science, Kwame Nkrumah University of Science and TechnologyDepartment of Computer Science, Kwame Nkrumah University of Science and TechnologyAbstract Research on sign language translation is ongoing with a high social inclusive goal of crossing the bridge between people with hearing disability using sign language as their basic way to communicate to others who do not understand sign language. Hundreds of different sign languages exist instead of a single universal sign language. Research on translating sign languages from high-income nations has grown significantly, but little is known about translating sign languages from Africa. In this paper, we curate a novel video-to-text African sign languages translation dataset containing sign language videos of Bible verses from six (6) different African countries. We experimented with competitive machine translation and sign language translation techniques on our dataset, including the application of transformers to sign language translation, multilingual training, and cross-transfer learning. We evaluated them in terms of accuracy and precision. The results from our experiments prove that having one Multilingual model for all the languages tends to be a better choice when deployed in real system in terms of memory usage with an accuracy of 94.6% and precision of 97.3%. These results give headway for more multilingual models to be developed to enhance inclusion for the deaf community and bridge the gap between the hearing and the deaf in Africa.https://doi.org/10.1007/s44163-025-00227-7Sign language translationAfrican sign languagesCross transfer learningHearing disabilityMultilingual trainingCompetitive machine translation |
| spellingShingle | Kate Takyi Rose-Mary Owusuaa Mensah Gyening Shester Msouobu Gueuwou Marco Stanley Nyarko Richard Adade Reindorf Nartey Borkor Samuelson Israel Boadu-Acheampong Linus Tabari AfriSign: African sign languages machine translation Discover Artificial Intelligence Sign language translation African sign languages Cross transfer learning Hearing disability Multilingual training Competitive machine translation |
| title | AfriSign: African sign languages machine translation |
| title_full | AfriSign: African sign languages machine translation |
| title_fullStr | AfriSign: African sign languages machine translation |
| title_full_unstemmed | AfriSign: African sign languages machine translation |
| title_short | AfriSign: African sign languages machine translation |
| title_sort | afrisign african sign languages machine translation |
| topic | Sign language translation African sign languages Cross transfer learning Hearing disability Multilingual training Competitive machine translation |
| url | https://doi.org/10.1007/s44163-025-00227-7 |
| work_keys_str_mv | AT katetakyi afrisignafricansignlanguagesmachinetranslation AT rosemaryowusuaamensahgyening afrisignafricansignlanguagesmachinetranslation AT shestermsouobugueuwou afrisignafricansignlanguagesmachinetranslation AT marcostanleynyarko afrisignafricansignlanguagesmachinetranslation AT richardadade afrisignafricansignlanguagesmachinetranslation AT reindorfnarteyborkor afrisignafricansignlanguagesmachinetranslation AT samuelsonisraelboaduacheampong afrisignafricansignlanguagesmachinetranslation AT linustabari afrisignafricansignlanguagesmachinetranslation |