American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks
Technological advancements play a significant role in the integration of deaf and mute individuals into society. Therefore, improvements in sign language recognition systems are of great importance. Many studies on sign languages have been conducted using real numbers. In this paper, a new approach...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10680527/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846108165816451072 |
|---|---|
| author | Selda Bayrak Vasif Nabiyev Celal Atalar |
| author_facet | Selda Bayrak Vasif Nabiyev Celal Atalar |
| author_sort | Selda Bayrak |
| collection | DOAJ |
| description | Technological advancements play a significant role in the integration of deaf and mute individuals into society. Therefore, improvements in sign language recognition systems are of great importance. Many studies on sign languages have been conducted using real numbers. In this paper, a new approach is presented for performing feature extraction from images and sign language alphabet recognition using complex numbers. In this context, a model is developed for recognizing American sign language. In the developed model, complex Zernike moments are used to obtain the feature vector of character images. A complex-valued deep neural network (CVDNN) capable of processing the feature vector composed of complex numbers across layers is also developed. CVDNNs are a powerful method capable of addressing the complex optimization issues of traditional deep neural networks more efficiently. CVDNNs, which use complex numbers as input data and complex activation functions in each layer, are expected to deliver superior performance in fields such as robotic systems, biometric technologies, disease diagnosis, and telecommunications. The model achieves recognition rates of 89.01% on the Sign Language MNIST dataset and 98.67% for holdout and 81.22% for leave-one-subject-out on the Massey University dataset, respectively, without any preprocessing. Our model, which is compared separately with many studies using the same datasets, shows the best performance when the two datasets are considered together. It has been observed that working with complex numbers resulted in a positive impact on performance of approximately 20% compared to configuring our model to work with real numbers while keeping its structure intact. |
| format | Article |
| id | doaj-art-c091d16dc0584f70b8ad94c2b3810249 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-c091d16dc0584f70b8ad94c2b38102492024-12-26T00:00:29ZengIEEEIEEE Access2169-35362024-01-011219300119301310.1109/ACCESS.2024.346157210680527American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural NetworksSelda Bayrak0https://orcid.org/0000-0003-0240-6592Vasif Nabiyev1https://orcid.org/0000-0003-0314-8134Celal Atalar2https://orcid.org/0000-0003-1932-5289Department of Software Engineering, Karadeniz Technical University, Trabzon, TürkiyeDepartment of Computer Engineering, Karadeniz Technical University, Trabzon, TürkiyeDepartment of Software Engineering, Karadeniz Technical University, Trabzon, TürkiyeTechnological advancements play a significant role in the integration of deaf and mute individuals into society. Therefore, improvements in sign language recognition systems are of great importance. Many studies on sign languages have been conducted using real numbers. In this paper, a new approach is presented for performing feature extraction from images and sign language alphabet recognition using complex numbers. In this context, a model is developed for recognizing American sign language. In the developed model, complex Zernike moments are used to obtain the feature vector of character images. A complex-valued deep neural network (CVDNN) capable of processing the feature vector composed of complex numbers across layers is also developed. CVDNNs are a powerful method capable of addressing the complex optimization issues of traditional deep neural networks more efficiently. CVDNNs, which use complex numbers as input data and complex activation functions in each layer, are expected to deliver superior performance in fields such as robotic systems, biometric technologies, disease diagnosis, and telecommunications. The model achieves recognition rates of 89.01% on the Sign Language MNIST dataset and 98.67% for holdout and 81.22% for leave-one-subject-out on the Massey University dataset, respectively, without any preprocessing. Our model, which is compared separately with many studies using the same datasets, shows the best performance when the two datasets are considered together. It has been observed that working with complex numbers resulted in a positive impact on performance of approximately 20% compared to configuring our model to work with real numbers while keeping its structure intact.https://ieeexplore.ieee.org/document/10680527/Complex valued deep neural networkcomplex Zernike momentsfeature extractionsign language recognition model |
| spellingShingle | Selda Bayrak Vasif Nabiyev Celal Atalar American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks IEEE Access Complex valued deep neural network complex Zernike moments feature extraction sign language recognition model |
| title | American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks |
| title_full | American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks |
| title_fullStr | American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks |
| title_full_unstemmed | American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks |
| title_short | American Sign Language Recognition Model Using Complex Zernike Moments and Complex-Valued Deep Neural Networks |
| title_sort | american sign language recognition model using complex zernike moments and complex valued deep neural networks |
| topic | Complex valued deep neural network complex Zernike moments feature extraction sign language recognition model |
| url | https://ieeexplore.ieee.org/document/10680527/ |
| work_keys_str_mv | AT seldabayrak americansignlanguagerecognitionmodelusingcomplexzernikemomentsandcomplexvalueddeepneuralnetworks AT vasifnabiyev americansignlanguagerecognitionmodelusingcomplexzernikemomentsandcomplexvalueddeepneuralnetworks AT celalatalar americansignlanguagerecognitionmodelusingcomplexzernikemomentsandcomplexvalueddeepneuralnetworks |