A Novel Sentence-Level Visual Speech Recognition System for Vietnamese Language Using ResNet3D and Zipformer
This paper presents the first sentence-level visual speech recognition (VSR) system specifically designed for the Vietnamese language. We have developed a unique dataset comprising 115 h of video recordings from over 100 speakers, focusing on single-speaker scenarios. The proposed VSR system utilize...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-01-01
|
| Series: | Modelling and Simulation in Engineering |
| Online Access: | http://dx.doi.org/10.1155/mse/2087573 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | This paper presents the first sentence-level visual speech recognition (VSR) system specifically designed for the Vietnamese language. We have developed a unique dataset comprising 115 h of video recordings from over 100 speakers, focusing on single-speaker scenarios. The proposed VSR system utilizes a ResNet3D architecture as the visual frontend, paired with a neural transducer framework featuring a Zipformer speech encoder. It incorporates a stateless decoder that considers two preceding tokens and is optimized with a pruned-RNNT loss function. Experimental results show that our system achieves a word error rate (WER) of 27.14% and a character error rate (CER) of 20.45% on single-speaker tasks, demonstrating significant progress in VSR for Vietnamese. |
|---|---|
| ISSN: | 1687-5605 |