Exploring MediaPipe optimization strategies for real-time sign language recognition
The present study meticulously investigates optimization strategies for real-time sign language recognition (SLR) employing the MediaPipe framework. We introduce an innovative multi-modal methodology, amalgamating four distinct Long Short-Term Memory (LSTM) models dedicated to processing skeletal c...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Can Tho University Publisher
2023-10-01
|
| Series: | CTU Journal of Innovation and Sustainable Development |
| Subjects: | |
| Online Access: | https://ctujs.ctu.edu.vn/index.php/ctujs/article/view/716 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849717596751921152 |
|---|---|
| author | Phuoc Thanh Nguyen Thanh Hoang Nguyen Ngoc Xuan Nguyen Hoang Huynh Thanh Binh Phan Hoang Son Hai Vu Hieu Nhan Huynh |
| author_facet | Phuoc Thanh Nguyen Thanh Hoang Nguyen Ngoc Xuan Nguyen Hoang Huynh Thanh Binh Phan Hoang Son Hai Vu Hieu Nhan Huynh |
| author_sort | Phuoc Thanh Nguyen |
| collection | DOAJ |
| description |
The present study meticulously investigates optimization strategies for real-time sign language recognition (SLR) employing the MediaPipe framework. We introduce an innovative multi-modal methodology, amalgamating four distinct Long Short-Term Memory (LSTM) models dedicated to processing skeletal coordinates ascertained from the MediaPipe framework. Rigorous evaluations were executed on esteemed sign language datasets. Empirical findings underscore that the multi-modal approach significantly elevates the accuracy of the SLR model while preserving its real-time capabilities. In comparative analyses with prevalent MediaPipe-based models, our multi-modal strategy consistently manifested superior performance metrics. A distinguishing characteristic of this approach is its inherent adaptability, facilitating modifications within the LSTM layers, rendering it apt for a myriad of challenges and data typologies. Integrating the MediaPipe framework with real-time SLR markedly amplifies recognition precision, signifying a pivotal advancement in the discipline.
|
| format | Article |
| id | doaj-art-c571062de5d9455e95c6d4646fb72f75 |
| institution | DOAJ |
| issn | 2588-1418 2815-6412 |
| language | English |
| publishDate | 2023-10-01 |
| publisher | Can Tho University Publisher |
| record_format | Article |
| series | CTU Journal of Innovation and Sustainable Development |
| spelling | doaj-art-c571062de5d9455e95c6d4646fb72f752025-08-20T03:12:36ZengCan Tho University PublisherCTU Journal of Innovation and Sustainable Development2588-14182815-64122023-10-0115Special issue: ISDS10.22144/ctujoisd.2023.045Exploring MediaPipe optimization strategies for real-time sign language recognitionPhuoc Thanh Nguyen0Thanh Hoang NguyenNgoc Xuan Nguyen HoangHuynh Thanh Binh PhanHoang Son Hai Vu Hieu Nhan Huynh 0911427905 The present study meticulously investigates optimization strategies for real-time sign language recognition (SLR) employing the MediaPipe framework. We introduce an innovative multi-modal methodology, amalgamating four distinct Long Short-Term Memory (LSTM) models dedicated to processing skeletal coordinates ascertained from the MediaPipe framework. Rigorous evaluations were executed on esteemed sign language datasets. Empirical findings underscore that the multi-modal approach significantly elevates the accuracy of the SLR model while preserving its real-time capabilities. In comparative analyses with prevalent MediaPipe-based models, our multi-modal strategy consistently manifested superior performance metrics. A distinguishing characteristic of this approach is its inherent adaptability, facilitating modifications within the LSTM layers, rendering it apt for a myriad of challenges and data typologies. Integrating the MediaPipe framework with real-time SLR markedly amplifies recognition precision, signifying a pivotal advancement in the discipline. https://ctujs.ctu.edu.vn/index.php/ctujs/article/view/716LSTM, MediaPipe, How2Sign, Indian Sign Language, ISL |
| spellingShingle | Phuoc Thanh Nguyen Thanh Hoang Nguyen Ngoc Xuan Nguyen Hoang Huynh Thanh Binh Phan Hoang Son Hai Vu Hieu Nhan Huynh Exploring MediaPipe optimization strategies for real-time sign language recognition CTU Journal of Innovation and Sustainable Development LSTM, MediaPipe, How2Sign, Indian Sign Language, ISL |
| title | Exploring MediaPipe optimization strategies for real-time sign language recognition |
| title_full | Exploring MediaPipe optimization strategies for real-time sign language recognition |
| title_fullStr | Exploring MediaPipe optimization strategies for real-time sign language recognition |
| title_full_unstemmed | Exploring MediaPipe optimization strategies for real-time sign language recognition |
| title_short | Exploring MediaPipe optimization strategies for real-time sign language recognition |
| title_sort | exploring mediapipe optimization strategies for real time sign language recognition |
| topic | LSTM, MediaPipe, How2Sign, Indian Sign Language, ISL |
| url | https://ctujs.ctu.edu.vn/index.php/ctujs/article/view/716 |
| work_keys_str_mv | AT phuocthanhnguyen exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition AT thanhhoangnguyen exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition AT ngocxuannguyenhoang exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition AT huynhthanhbinhphan exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition AT hoangsonhaivu exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition AT hieunhanhuynh exploringmediapipeoptimizationstrategiesforrealtimesignlanguagerecognition |