An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models
Hearing-impaired individuals face significant challenges in social interactions due to communication barriers. Advances in technology have introduced numerous assistive tools to bridge this gap. This research aims to enhance communication between hearing-impaired and hearing individuals by developin...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10778485/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850254939852374016 |
|---|---|
| author | Mogeeb A. A. Mosleh Abdu H. Gumaei |
| author_facet | Mogeeb A. A. Mosleh Abdu H. Gumaei |
| author_sort | Mogeeb A. A. Mosleh |
| collection | DOAJ |
| description | Hearing-impaired individuals face significant challenges in social interactions due to communication barriers. Advances in technology have introduced numerous assistive tools to bridge this gap. This research aims to enhance communication between hearing-impaired and hearing individuals by developing a translation system for Yemeni Arabic Sign Language. The prototype leverages CNN deep learning models and fuzzy string matching to translate signs and text efficiently. The Yemeni Sign Language dataset comprises 24,245 images representing 32 commonly used words. The prototype consists of two primary modules for translating text to sign language and vice versa in both directions. The initial prototype module was created to convert sign language input from hearing-impaired individuals into text via five CNN transfer learning models: MobileNet, GoogleNet, VGG16, ResNet152, and DenseNet161. Fuzzy string-matching and data sign dictionary approaches were utilized to convert the input words into sign images. The proposed system achieved high translation accuracy rates for various CNN models, with ResNet152 scoring 98.78%, MobileNet scoring 97.94%, GoogleNet scoring 98.36%, VGG16 scoring 90.46%, and DenseNet161 scoring 98.34% based on the experimental results. Furthermore, the experimental data demonstrated that fuzzy matching score models can effectively convert the input word into a hand sign image with excellent performance. The fuzzy matching score model efficiently solves the issues of synonym words and spelling typo errors and provides a fast translation approach. The suggested model demonstrated the ability to create a bidirectional sign language translation Android application for Yemeni Arabic sign language with outstanding accuracy and performance. It also has the capacity to create a robust bidirectional Arabic sign language translation system through the utilization of deep learning techniques. |
| format | Article |
| id | doaj-art-23c5f0e76465453cbcedd35a9c539a52 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-23c5f0e76465453cbcedd35a9c539a522025-08-20T01:57:00ZengIEEEIEEE Access2169-35362024-01-011219103019104510.1109/ACCESS.2024.351245510778485An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning ModelsMogeeb A. A. Mosleh0https://orcid.org/0000-0001-5094-5561Abdu H. Gumaei1Software Engineering Department, Faculty of Engineering and Information Technology, Taiz University, Taizz, YemenSoftware Engineering Department, Faculty of Engineering and Information Technology, Taiz University, Taizz, YemenHearing-impaired individuals face significant challenges in social interactions due to communication barriers. Advances in technology have introduced numerous assistive tools to bridge this gap. This research aims to enhance communication between hearing-impaired and hearing individuals by developing a translation system for Yemeni Arabic Sign Language. The prototype leverages CNN deep learning models and fuzzy string matching to translate signs and text efficiently. The Yemeni Sign Language dataset comprises 24,245 images representing 32 commonly used words. The prototype consists of two primary modules for translating text to sign language and vice versa in both directions. The initial prototype module was created to convert sign language input from hearing-impaired individuals into text via five CNN transfer learning models: MobileNet, GoogleNet, VGG16, ResNet152, and DenseNet161. Fuzzy string-matching and data sign dictionary approaches were utilized to convert the input words into sign images. The proposed system achieved high translation accuracy rates for various CNN models, with ResNet152 scoring 98.78%, MobileNet scoring 97.94%, GoogleNet scoring 98.36%, VGG16 scoring 90.46%, and DenseNet161 scoring 98.34% based on the experimental results. Furthermore, the experimental data demonstrated that fuzzy matching score models can effectively convert the input word into a hand sign image with excellent performance. The fuzzy matching score model efficiently solves the issues of synonym words and spelling typo errors and provides a fast translation approach. The suggested model demonstrated the ability to create a bidirectional sign language translation Android application for Yemeni Arabic sign language with outstanding accuracy and performance. It also has the capacity to create a robust bidirectional Arabic sign language translation system through the utilization of deep learning techniques.https://ieeexplore.ieee.org/document/10778485/Yemini Arabic sign languagetransfer learningfuzzy score matchingCNN models |
| spellingShingle | Mogeeb A. A. Mosleh Abdu H. Gumaei An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models IEEE Access Yemini Arabic sign language transfer learning fuzzy score matching CNN models |
| title | An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models |
| title_full | An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models |
| title_fullStr | An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models |
| title_full_unstemmed | An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models |
| title_short | An Efficient Bidirectional Android Translation Prototype for Yemeni Sign Language Using Fuzzy Logic and CNN Transfer Learning Models |
| title_sort | efficient bidirectional android translation prototype for yemeni sign language using fuzzy logic and cnn transfer learning models |
| topic | Yemini Arabic sign language transfer learning fuzzy score matching CNN models |
| url | https://ieeexplore.ieee.org/document/10778485/ |
| work_keys_str_mv | AT mogeebaamosleh anefficientbidirectionalandroidtranslationprototypeforyemenisignlanguageusingfuzzylogicandcnntransferlearningmodels AT abduhgumaei anefficientbidirectionalandroidtranslationprototypeforyemenisignlanguageusingfuzzylogicandcnntransferlearningmodels AT mogeebaamosleh efficientbidirectionalandroidtranslationprototypeforyemenisignlanguageusingfuzzylogicandcnntransferlearningmodels AT abduhgumaei efficientbidirectionalandroidtranslationprototypeforyemenisignlanguageusingfuzzylogicandcnntransferlearningmodels |