LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word
Sign language plays a critical role in enabling communication for the Deaf and hard-of-hearing community in Indonesia, yet there remains a significant gap in technological support for recognizing the official Indonesian sign language, Sistem Isyarat Bahasa Indonesia (SIBI). This study presents a dee...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Politeknik Negeri Batam
2025-06-01
|
| Series: | Journal of Applied Informatics and Computing |
| Subjects: | |
| Online Access: | https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/9607 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849729395858604032 |
|---|---|
| author | Patricia Ho Handri Santoso |
| author_facet | Patricia Ho Handri Santoso |
| author_sort | Patricia Ho |
| collection | DOAJ |
| description | Sign language plays a critical role in enabling communication for the Deaf and hard-of-hearing community in Indonesia, yet there remains a significant gap in technological support for recognizing the official Indonesian sign language, Sistem Isyarat Bahasa Indonesia (SIBI). This study presents a deep learning-based hand gesture recognition system for SIBI, focusing on four primary gesture categories: affix, alphabet, number, and word. A large and diverse dataset of 21,351 videos was collected, covering 18 affix, 26 alphabet, 35 number, and 29 word classes. Hand keypoints were extracted using MediaPipe Holistic, and a bidirectional long short-term memory (BiLSTM) model was trained using 5-fold stratified cross-validation. The model achieved high recognition performance in the alphabet, number, and word categories, with mean test accuracies of 93.94%, 91.48%, and 92.41%, respectively, and slightly lower performance in the affix category at 68.17%. The affix category posed particular challenges due to subtle hand shape differences and high variability between signers, while the alphabet category consistently showed the highest accuracy due to its distinct and static handshapes. Evaluation metrics, including precision, recall, F1-score, and confusion matrix analysis, provided further insights into model strengths and limitations. Overall, the study demonstrates the effectiveness of LSTM models for sequential hand gesture recognition in SIBI and highlights areas for future improvement, such as handling non-manual features and improving generalization across signers. |
| format | Article |
| id | doaj-art-627004bdf5a44fbd9a51e7f516de2ced |
| institution | DOAJ |
| issn | 2548-6861 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Politeknik Negeri Batam |
| record_format | Article |
| series | Journal of Applied Informatics and Computing |
| spelling | doaj-art-627004bdf5a44fbd9a51e7f516de2ced2025-08-20T03:09:13ZengPoliteknik Negeri BatamJournal of Applied Informatics and Computing2548-68612025-06-019392893710.30871/jaic.v9i3.96077152LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and WordPatricia Ho0Handri Santoso1Pradita UniversityPradita UniversitySign language plays a critical role in enabling communication for the Deaf and hard-of-hearing community in Indonesia, yet there remains a significant gap in technological support for recognizing the official Indonesian sign language, Sistem Isyarat Bahasa Indonesia (SIBI). This study presents a deep learning-based hand gesture recognition system for SIBI, focusing on four primary gesture categories: affix, alphabet, number, and word. A large and diverse dataset of 21,351 videos was collected, covering 18 affix, 26 alphabet, 35 number, and 29 word classes. Hand keypoints were extracted using MediaPipe Holistic, and a bidirectional long short-term memory (BiLSTM) model was trained using 5-fold stratified cross-validation. The model achieved high recognition performance in the alphabet, number, and word categories, with mean test accuracies of 93.94%, 91.48%, and 92.41%, respectively, and slightly lower performance in the affix category at 68.17%. The affix category posed particular challenges due to subtle hand shape differences and high variability between signers, while the alphabet category consistently showed the highest accuracy due to its distinct and static handshapes. Evaluation metrics, including precision, recall, F1-score, and confusion matrix analysis, provided further insights into model strengths and limitations. Overall, the study demonstrates the effectiveness of LSTM models for sequential hand gesture recognition in SIBI and highlights areas for future improvement, such as handling non-manual features and improving generalization across signers.https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/9607deep learninghand gesture recognitionindonesian sign language system (sibi)lstm |
| spellingShingle | Patricia Ho Handri Santoso LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word Journal of Applied Informatics and Computing deep learning hand gesture recognition indonesian sign language system (sibi) lstm |
| title | LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word |
| title_full | LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word |
| title_fullStr | LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word |
| title_full_unstemmed | LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word |
| title_short | LSTM-Based Hand Gesture Recognition for Indonesian Sign Language System (SIBI) on Affix, Alphabet, Number, and Word |
| title_sort | lstm based hand gesture recognition for indonesian sign language system sibi on affix alphabet number and word |
| topic | deep learning hand gesture recognition indonesian sign language system (sibi) lstm |
| url | https://jurnal.polibatam.ac.id/index.php/JAIC/article/view/9607 |
| work_keys_str_mv | AT patriciaho lstmbasedhandgesturerecognitionforindonesiansignlanguagesystemsibionaffixalphabetnumberandword AT handrisantoso lstmbasedhandgesturerecognitionforindonesiansignlanguagesystemsibionaffixalphabetnumberandword |