Machine learning-assisted wearable sensing systems for speech recognition and interaction
Abstract The human voice stands out for its rich information transmission capabilities. However, voice communication is susceptible to interference from noisy environments and obstacles. Here, we propose a wearable wireless flexible skin-attached acoustic sensor (SAAS) capable of capturing the vibra...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-03-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-57629-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850040173981597696 |
|---|---|
| author | Tao Liu Mingyang Zhang Zhihao Li Hanjie Dou Wangyang Zhang Jiaqian Yang Pengfan Wu Dongxiao Li Xiaojing Mu |
| author_facet | Tao Liu Mingyang Zhang Zhihao Li Hanjie Dou Wangyang Zhang Jiaqian Yang Pengfan Wu Dongxiao Li Xiaojing Mu |
| author_sort | Tao Liu |
| collection | DOAJ |
| description | Abstract The human voice stands out for its rich information transmission capabilities. However, voice communication is susceptible to interference from noisy environments and obstacles. Here, we propose a wearable wireless flexible skin-attached acoustic sensor (SAAS) capable of capturing the vibrations of vocal organs and skin movements, thereby enabling voice recognition and human-machine interaction (HMI) in harsh acoustic environments. This system utilizes a piezoelectric micromachined ultrasonic transducers (PMUT), which feature high sensitivity (-198 dB), wide bandwidth (10 Hz-20 kHz), and excellent flatness (±0.5 dB). Flexible packaging enhances comfort and adaptability during wear, while integration with the Residual Network (ResNet) architecture significantly improves the classification of laryngeal speech features, achieving an accuracy exceeding 96%. Furthermore, we also demonstrated SAAS’s data collection and intelligent classification capabilities in multiple HMI scenarios. Finally, the speech recognition system was able to recognize everyday sentences spoken by participants with an accuracy of 99.8% through a deep learning model. With advantages including a simple fabrication process, stable performance, easy integration, and low cost, SAAS presents a compelling solution for applications in voice control, HMI, and wearable electronics. |
| format | Article |
| id | doaj-art-9ab9c3ef1b304bc6852d57cd7f2d6ab5 |
| institution | DOAJ |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-9ab9c3ef1b304bc6852d57cd7f2d6ab52025-08-20T02:56:09ZengNature PortfolioNature Communications2041-17232025-03-0116111310.1038/s41467-025-57629-5Machine learning-assisted wearable sensing systems for speech recognition and interactionTao Liu0Mingyang Zhang1Zhihao Li2Hanjie Dou3Wangyang Zhang4Jiaqian Yang5Pengfan Wu6Dongxiao Li7Xiaojing Mu8Key Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityKey Laboratory of Optoelectronic Technology & Systems of Ministry of Education, International R & D Center of Micro-nano Systems and New Materials Technology, Chongqing UniversityAbstract The human voice stands out for its rich information transmission capabilities. However, voice communication is susceptible to interference from noisy environments and obstacles. Here, we propose a wearable wireless flexible skin-attached acoustic sensor (SAAS) capable of capturing the vibrations of vocal organs and skin movements, thereby enabling voice recognition and human-machine interaction (HMI) in harsh acoustic environments. This system utilizes a piezoelectric micromachined ultrasonic transducers (PMUT), which feature high sensitivity (-198 dB), wide bandwidth (10 Hz-20 kHz), and excellent flatness (±0.5 dB). Flexible packaging enhances comfort and adaptability during wear, while integration with the Residual Network (ResNet) architecture significantly improves the classification of laryngeal speech features, achieving an accuracy exceeding 96%. Furthermore, we also demonstrated SAAS’s data collection and intelligent classification capabilities in multiple HMI scenarios. Finally, the speech recognition system was able to recognize everyday sentences spoken by participants with an accuracy of 99.8% through a deep learning model. With advantages including a simple fabrication process, stable performance, easy integration, and low cost, SAAS presents a compelling solution for applications in voice control, HMI, and wearable electronics.https://doi.org/10.1038/s41467-025-57629-5 |
| spellingShingle | Tao Liu Mingyang Zhang Zhihao Li Hanjie Dou Wangyang Zhang Jiaqian Yang Pengfan Wu Dongxiao Li Xiaojing Mu Machine learning-assisted wearable sensing systems for speech recognition and interaction Nature Communications |
| title | Machine learning-assisted wearable sensing systems for speech recognition and interaction |
| title_full | Machine learning-assisted wearable sensing systems for speech recognition and interaction |
| title_fullStr | Machine learning-assisted wearable sensing systems for speech recognition and interaction |
| title_full_unstemmed | Machine learning-assisted wearable sensing systems for speech recognition and interaction |
| title_short | Machine learning-assisted wearable sensing systems for speech recognition and interaction |
| title_sort | machine learning assisted wearable sensing systems for speech recognition and interaction |
| url | https://doi.org/10.1038/s41467-025-57629-5 |
| work_keys_str_mv | AT taoliu machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT mingyangzhang machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT zhihaoli machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT hanjiedou machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT wangyangzhang machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT jiaqianyang machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT pengfanwu machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT dongxiaoli machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction AT xiaojingmu machinelearningassistedwearablesensingsystemsforspeechrecognitionandinteraction |