Nonlinear multi-head cross-attention network and programmable gradient information for gaze estimation

Nonlinear multi-head cross-attention network and programmable gradient information for gaze estimation

Abstract Gaze estimation is an important indicator of human behavior that can be used for human assistance. Recent gaze estimation methods are primarily based on convolutional neural networks (CNNs) or attention Transformers. However, CNNs extract a limited local context while losing important globa...

Full description

Saved in:
Bibliographic Details
Main Authors: Yujie Li, Yuhang Hong, Ziwen Wang, Jiahui Chen, Rongjie Liu, Shuxue Ding, Benying Tan
Format: Article
Language:English
Published: Nature Portfolio 2025-07-01
Series:Scientific Reports
Subjects:
Gaze estimation
Convolutional neural network
Transformer
Programmable gradient information
Online Access:https://doi.org/10.1038/s41598-025-12466-w
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Abstract Gaze estimation is an important indicator of human behavior that can be used for human assistance. Recent gaze estimation methods are primarily based on convolutional neural networks (CNNs) or attention Transformers. However, CNNs extract a limited local context while losing important global information, whereas attention mechanisms exhibit low utilization of multiscale hybrid features. To address these issues, we propose a novel nonlinear multi-head cross-attention network with programmable gradient information (MCA-PGI), which synthesizes the advantages of CNNs and the Transformer. The programmable gradient information is used to achieve reliable gradient propagation. An auxiliary branch is incorporated to integrate the gradient information, thereby retaining more original information than CNNs. In addition, nonlinear multi-head cross-attention is employed to fuse the global visual and multiscale hybrid features for more accurate gaze estimation. Experimental results on three publicly available datasets demonstrate that the proposed MCA-PGI exhibits strong competitiveness and outperforms most state-of-the-art methods, achieving 2.5% and 10.2% performance improvements on the MPIIFaceGaze and Eyediap datasets, respectively. The implementation code can be found at https://github.com/Yuhang-Hong/MCA-PGI .
ISSN:2045-2322

Similar Items