Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition
Sensor-based gesture recognition on mobile devices is critical to human–computer interaction, enabling intuitive user input for various applications. However, current approaches often rely on server-based retraining whenever new gestures are introduced, incurring substantial energy consumption and l...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/2/427 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832587504422748160 |
|---|---|
| author | Heon-Sung Park Min-Kyung Sung Dae-Won Kim Jaesung Lee |
| author_facet | Heon-Sung Park Min-Kyung Sung Dae-Won Kim Jaesung Lee |
| author_sort | Heon-Sung Park |
| collection | DOAJ |
| description | Sensor-based gesture recognition on mobile devices is critical to human–computer interaction, enabling intuitive user input for various applications. However, current approaches often rely on server-based retraining whenever new gestures are introduced, incurring substantial energy consumption and latency due to frequent data transmission. To address these limitations, we present the first on-device continual learning framework for gesture recognition. Leveraging the Nearest Class Mean (NCM) classifier coupled with a replay-based update strategy, our method enables continuous adaptation to new gestures under limited computing and memory resources. By employing replay buffer management, we efficiently store and revisit previously learned instances, mitigating catastrophic forgetting and ensuring stable performance as new gestures are added. Experimental results on a Samsung Galaxy S10 device demonstrate that our method achieves over 99% accuracy while operating entirely on-device, offering a compelling synergy between computational efficiency, robust continual learning, and high recognition accuracy. This work demonstrates the potential of on-device continual learning frameworks that integrate NCM classifiers with replay-based techniques, thereby advancing the field of resource-constrained, adaptive gesture recognition. |
| format | Article |
| id | doaj-art-7ed87b582ba2404292c7969a53a3e953 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-7ed87b582ba2404292c7969a53a3e9532025-01-24T13:48:54ZengMDPI AGSensors1424-82202025-01-0125242710.3390/s25020427Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture RecognitionHeon-Sung Park0Min-Kyung Sung1Dae-Won Kim2Jaesung Lee3School of Computer Science and Engineering, Chung-Dang University, Heukseok-dong, Dongjak-gu, Seoul 06974, Republic of KoreaDepartment of Artifcial Intelligence, Chung-Ang University, Heukseok-dong, Dongjak-gu, Seoul 06974, Republic of KoreaSchool of Computer Science and Engineering, Chung-Dang University, Heukseok-dong, Dongjak-gu, Seoul 06974, Republic of KoreaDepartment of Artifcial Intelligence, Chung-Ang University, Heukseok-dong, Dongjak-gu, Seoul 06974, Republic of KoreaSensor-based gesture recognition on mobile devices is critical to human–computer interaction, enabling intuitive user input for various applications. However, current approaches often rely on server-based retraining whenever new gestures are introduced, incurring substantial energy consumption and latency due to frequent data transmission. To address these limitations, we present the first on-device continual learning framework for gesture recognition. Leveraging the Nearest Class Mean (NCM) classifier coupled with a replay-based update strategy, our method enables continuous adaptation to new gestures under limited computing and memory resources. By employing replay buffer management, we efficiently store and revisit previously learned instances, mitigating catastrophic forgetting and ensuring stable performance as new gestures are added. Experimental results on a Samsung Galaxy S10 device demonstrate that our method achieves over 99% accuracy while operating entirely on-device, offering a compelling synergy between computational efficiency, robust continual learning, and high recognition accuracy. This work demonstrates the potential of on-device continual learning frameworks that integrate NCM classifiers with replay-based techniques, thereby advancing the field of resource-constrained, adaptive gesture recognition.https://www.mdpi.com/1424-8220/25/2/427gesture recognitionon-device AIcontinual learning |
| spellingShingle | Heon-Sung Park Min-Kyung Sung Dae-Won Kim Jaesung Lee Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition Sensors gesture recognition on-device AI continual learning |
| title | Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition |
| title_full | Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition |
| title_fullStr | Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition |
| title_full_unstemmed | Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition |
| title_short | Real-Time On-Device Continual Learning Based on a Combined Nearest Class Mean and Replay Method for Smartphone Gesture Recognition |
| title_sort | real time on device continual learning based on a combined nearest class mean and replay method for smartphone gesture recognition |
| topic | gesture recognition on-device AI continual learning |
| url | https://www.mdpi.com/1424-8220/25/2/427 |
| work_keys_str_mv | AT heonsungpark realtimeondevicecontinuallearningbasedonacombinednearestclassmeanandreplaymethodforsmartphonegesturerecognition AT minkyungsung realtimeondevicecontinuallearningbasedonacombinednearestclassmeanandreplaymethodforsmartphonegesturerecognition AT daewonkim realtimeondevicecontinuallearningbasedonacombinednearestclassmeanandreplaymethodforsmartphonegesturerecognition AT jaesunglee realtimeondevicecontinuallearningbasedonacombinednearestclassmeanandreplaymethodforsmartphonegesturerecognition |