Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism
Hand gesture recognition (HGR) is a field of action recognition widely used in various domains such as robotics, virtual reality (VR), and augmented reality (AR). In this paper, we propose a semantically interpretable attention technique based on the compression and exchange of local and global info...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10879500/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850191389057351680 |
|---|---|
| author | Moon Ju Chae Sang Hoon Han Hyeok Nam Jae Hyeon Park Min Hee Cha Sung In Cho |
| author_facet | Moon Ju Chae Sang Hoon Han Hyeok Nam Jae Hyeon Park Min Hee Cha Sung In Cho |
| author_sort | Moon Ju Chae |
| collection | DOAJ |
| description | Hand gesture recognition (HGR) is a field of action recognition widely used in various domains such as robotics, virtual reality (VR), and augmented reality (AR). In this paper, we propose a semantically interpretable attention technique based on the compression and exchange of local and global information for real-time dynamic hand gesture recognition. In this research, we focus on data comprising hand landmark coordinates and online recognition of multiple gestures within a single sequence. Specifically, our approach has two paths to learn intraframe and interframe information separately. The learned information is compressed in the local and global perspectives, and the compressed information is exchanged through cross-attention. By using this approach, the importance of each hand landmark and frame, which can be interpreted semantically, can be extracted, and this information is used in the attention process on the intraframe and interframe information. Finally, the intraframe and interframe information to which attention is applied is integrated, which effectively enables comprehensive feature extraction of both local and global information. Experimental results demonstrated that the proposed method enabled concise and rapid hand-gesture recognition. It provided 95% accuracy in real-time hand-gesture recognition on a SHREC’22 dataset and accurately estimated the conclusion of a given gesture. Additionally, with a speed of approximately 294 frames per second (FPS), our model is well-suited for real-time systems, offering users immersive experience. This demonstrates its potential for effective application in real-world environments. |
| format | Article |
| id | doaj-art-ffa145b59ec74f238b9e5775275a7f75 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-ffa145b59ec74f238b9e5775275a7f752025-08-20T02:14:56ZengIEEEIEEE Access2169-35362025-01-0113323293234010.1109/ACCESS.2025.354072110879500Online Hand Gesture Recognition Using Semantically Interpretable Attention MechanismMoon Ju Chae0https://orcid.org/0009-0006-7195-1454Sang Hoon Han1https://orcid.org/0009-0005-7755-9587Hyeok Nam2https://orcid.org/0009-0008-6301-1782Jae Hyeon Park3https://orcid.org/0000-0002-6233-4394Min Hee Cha4Sung In Cho5https://orcid.org/0000-0003-4251-7131Department of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaDepartment of Multimedia Engineering, Dongguk University, Seoul, Republic of KoreaHand gesture recognition (HGR) is a field of action recognition widely used in various domains such as robotics, virtual reality (VR), and augmented reality (AR). In this paper, we propose a semantically interpretable attention technique based on the compression and exchange of local and global information for real-time dynamic hand gesture recognition. In this research, we focus on data comprising hand landmark coordinates and online recognition of multiple gestures within a single sequence. Specifically, our approach has two paths to learn intraframe and interframe information separately. The learned information is compressed in the local and global perspectives, and the compressed information is exchanged through cross-attention. By using this approach, the importance of each hand landmark and frame, which can be interpreted semantically, can be extracted, and this information is used in the attention process on the intraframe and interframe information. Finally, the intraframe and interframe information to which attention is applied is integrated, which effectively enables comprehensive feature extraction of both local and global information. Experimental results demonstrated that the proposed method enabled concise and rapid hand-gesture recognition. It provided 95% accuracy in real-time hand-gesture recognition on a SHREC’22 dataset and accurately estimated the conclusion of a given gesture. Additionally, with a speed of approximately 294 frames per second (FPS), our model is well-suited for real-time systems, offering users immersive experience. This demonstrates its potential for effective application in real-world environments.https://ieeexplore.ieee.org/document/10879500/Hand gesture recognitiononline recognitionintraframe and interframe informationcross-attention |
| spellingShingle | Moon Ju Chae Sang Hoon Han Hyeok Nam Jae Hyeon Park Min Hee Cha Sung In Cho Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism IEEE Access Hand gesture recognition online recognition intraframe and interframe information cross-attention |
| title | Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism |
| title_full | Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism |
| title_fullStr | Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism |
| title_full_unstemmed | Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism |
| title_short | Online Hand Gesture Recognition Using Semantically Interpretable Attention Mechanism |
| title_sort | online hand gesture recognition using semantically interpretable attention mechanism |
| topic | Hand gesture recognition online recognition intraframe and interframe information cross-attention |
| url | https://ieeexplore.ieee.org/document/10879500/ |
| work_keys_str_mv | AT moonjuchae onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism AT sanghoonhan onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism AT hyeoknam onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism AT jaehyeonpark onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism AT minheecha onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism AT sungincho onlinehandgesturerecognitionusingsemanticallyinterpretableattentionmechanism |