A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language
Many isolated words in Chinese Sign Language (CSL) exhibit significant feature similarities, which are primarily conveyed through hand, face, and body movements. Among these, hand features are particularly crucial, as they carry substantial information about the sign language words. However, the spa...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10965662/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850191732623278080 |
|---|---|
| author | Yuxin Han Yong Han Qi Jiang |
| author_facet | Yuxin Han Yong Han Qi Jiang |
| author_sort | Yuxin Han |
| collection | DOAJ |
| description | Many isolated words in Chinese Sign Language (CSL) exhibit significant feature similarities, which are primarily conveyed through hand, face, and body movements. Among these, hand features are particularly crucial, as they carry substantial information about the sign language words. However, the spatial and temporal dimensions involved in sign language expressions are often relatively small, leading to a similarity problem that increases the difficulty of sign language recognition. The general extraction and recognition of sign language features can result in confusion when distinguishing similar words, and fine-grained feature extraction of sign language actions may incur high computational costs. To deal with these challenges, this paper proposes a dual-stream deep learning model built on Spatio-Temporal Graph Convolutional Network-Long Short-Term Memory(STGCN-LSTM), which aims to capture both the local features of sign language and the global spatio-temporal characteristics of sign words. The model focuses on capturing both local features in sign language and the overall spatio-temporal characteristics of sign language words. By learning four key sign language phonetic features—hand shape, hand position, hand orientation, and hand motion trajectory—as well as spatio-temporal features derived from whole-body skeletal data, the model aims to improve the recognition of Chinese Sign Language. In this paper, the effectiveness of the proposed model is validated on the SRL500 dataset and the sub-dataset of similar SRL500 consisting of similar words selected from it, and the recognition accuracies obtained are 95.2% and 93.0%, respectively. |
| format | Article |
| id | doaj-art-e9a7523aad774150ae461824705a1bc3 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e9a7523aad774150ae461824705a1bc32025-08-20T02:14:49ZengIEEEIEEE Access2169-35362025-01-0113748117482010.1109/ACCESS.2025.356077910965662A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign LanguageYuxin Han0https://orcid.org/0009-0002-8571-766XYong Han1https://orcid.org/0000-0001-9564-1668Qi Jiang2Faculty of Information Science and Engineering, Ocean University of China, Qingdao, Shandong, ChinaFaculty of Information Science and Engineering, Ocean University of China, Qingdao, Shandong, ChinaFaculty of Information Science and Engineering, Ocean University of China, Qingdao, Shandong, ChinaMany isolated words in Chinese Sign Language (CSL) exhibit significant feature similarities, which are primarily conveyed through hand, face, and body movements. Among these, hand features are particularly crucial, as they carry substantial information about the sign language words. However, the spatial and temporal dimensions involved in sign language expressions are often relatively small, leading to a similarity problem that increases the difficulty of sign language recognition. The general extraction and recognition of sign language features can result in confusion when distinguishing similar words, and fine-grained feature extraction of sign language actions may incur high computational costs. To deal with these challenges, this paper proposes a dual-stream deep learning model built on Spatio-Temporal Graph Convolutional Network-Long Short-Term Memory(STGCN-LSTM), which aims to capture both the local features of sign language and the global spatio-temporal characteristics of sign words. The model focuses on capturing both local features in sign language and the overall spatio-temporal characteristics of sign language words. By learning four key sign language phonetic features—hand shape, hand position, hand orientation, and hand motion trajectory—as well as spatio-temporal features derived from whole-body skeletal data, the model aims to improve the recognition of Chinese Sign Language. In this paper, the effectiveness of the proposed model is validated on the SRL500 dataset and the sub-dataset of similar SRL500 consisting of similar words selected from it, and the recognition accuracies obtained are 95.2% and 93.0%, respectively.https://ieeexplore.ieee.org/document/10965662/Chinese sign languagephonological featureshand gesture recognitionSTGCNConv-LSTM |
| spellingShingle | Yuxin Han Yong Han Qi Jiang A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language IEEE Access Chinese sign language phonological features hand gesture recognition STGCN Conv-LSTM |
| title | A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language |
| title_full | A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language |
| title_fullStr | A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language |
| title_full_unstemmed | A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language |
| title_short | A Study on the STGCN-LSTM Sign Language Recognition Model Based on Phonological Features of Sign Language |
| title_sort | study on the stgcn lstm sign language recognition model based on phonological features of sign language |
| topic | Chinese sign language phonological features hand gesture recognition STGCN Conv-LSTM |
| url | https://ieeexplore.ieee.org/document/10965662/ |
| work_keys_str_mv | AT yuxinhan astudyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage AT yonghan astudyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage AT qijiang astudyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage AT yuxinhan studyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage AT yonghan studyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage AT qijiang studyonthestgcnlstmsignlanguagerecognitionmodelbasedonphonologicalfeaturesofsignlanguage |