Hardware Efficient Speech Enhancement With Noise Aware Multi-Target Deep Learning

Hardware Efficient Speech Enhancement With Noise Aware Multi-Target Deep Learning

This paper describes a supervised speech enhancement (SE) method utilising a noise-aware four-layer deep neural network and training target switching. For optimal speech denoising, the SE system, trained with multiple-target joint learning, switches between mapping-based, masking-based, or complemen...

Full description

Saved in:
Bibliographic Details
Main Authors: Salinna Abdullah, Majid Zamani, Andreas Demosthenous
Format: Article
Language:English
Published: IEEE 2024-01-01
Series:IEEE Open Journal of Circuits and Systems
Subjects:
Deep neural network
digital circuits
field programmable gate array (FPGA)
mapping
masking
multi-target learning
Online Access:https://ieeexplore.ieee.org/document/10500889/
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper describes a supervised speech enhancement (SE) method utilising a noise-aware four-layer deep neural network and training target switching. For optimal speech denoising, the SE system, trained with multiple-target joint learning, switches between mapping-based, masking-based, or complementary processing, depending on the level of noise contamination detected. Optimisation techniques, including ternary quantisation, structural pruning, efficient sparse matrix representation and cost-effective approximations for complex computations, were implemented to reduce area, memory, and power requirements. Up to 19.1x compression was obtained, and all weights could be stored on the on-chip memory. When processing NOISEX-92 noises, the system achieved an average short-time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ) scores of 0.81 and 1.62, respectively, outperforming SE algorithms trained with only a single learning target. The proposed SE processor was implemented on a field programmable gate array (FPGA) for proof of concept. Mapping the design on a 65-nm CMOS process led to a chip core area of <inline-formula> <tex-math notation="LaTeX">$3.88~mm^{2}$ </tex-math></inline-formula> and a power consumption of 1.91 mW when operating at a 10 MHz clock frequency.
ISSN:2644-1225

Similar Items