HFA-Net: hierarchical feature aggregation network for micro-expression recognition
Abstract Micro-expressions (MEs) are unconscious and involuntary reactions that genuinely reflect an individual’s inner emotional state, making them valuable in the fields of emotion analysis and behavior recognition. MEs are characterized by subtle changes within specific facial action units, and e...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-02-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-025-01804-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850252256713113600 |
|---|---|
| author | Meng Zhang Wenzhong Yang Liejun Wang Zhonghua Wu Danny Chen |
| author_facet | Meng Zhang Wenzhong Yang Liejun Wang Zhonghua Wu Danny Chen |
| author_sort | Meng Zhang |
| collection | DOAJ |
| description | Abstract Micro-expressions (MEs) are unconscious and involuntary reactions that genuinely reflect an individual’s inner emotional state, making them valuable in the fields of emotion analysis and behavior recognition. MEs are characterized by subtle changes within specific facial action units, and effective feature learning and fusion tailored to these characteristics still require in-depth research. To address this challenge, this paper proposes a novel hierarchical feature aggregation network (HFA-Net). In the local branch, the multi-scale attention (MSA) block is proposed to capture subtle facial changes and local information. The global branch introduces the retentive meet transformers (RMT) block to establish dependencies between holistic facial features and structural information. Considering that single-scale features are insufficient to fully capture the subtleties of MEs, a multi-level feature aggregation (MLFA) module is proposed to extract and fuse features from different levels across the two branches, preserving more comprehensive feature information. To enhance the representation of key features, an adaptive attention feature fusion (AAFF) module is designed to focus on the most useful and relevant feature channels. Extensive experiments conducted on the SMIC, CASME II, and SAMM benchmark databases demonstrate that the proposed HFA-Net outperforms current state-of-the-art methods. Additionally, ablation studies confirm the superior discriminative capability of HFA-Net when learning feature representations from limited ME samples. Our code is publicly available at https://github.com/tairuwu/HFA-Net . |
| format | Article |
| id | doaj-art-e8e1ffccb8324fb991d5b85109d89670 |
| institution | OA Journals |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-e8e1ffccb8324fb991d5b85109d896702025-08-20T01:57:40ZengSpringerComplex & Intelligent Systems2199-45362198-60532025-02-0111312010.1007/s40747-025-01804-0HFA-Net: hierarchical feature aggregation network for micro-expression recognitionMeng Zhang0Wenzhong Yang1Liejun Wang2Zhonghua Wu3Danny Chen4School of Computer Science and Technology, Xinjiang UniversitySchool of Computer Science and Technology, Xinjiang UniversitySchool of Computer Science and Technology, Xinjiang UniversitySchool of Computer Science and Technology, Xinjiang UniversitySchool of Computer Science and Technology, Xinjiang UniversityAbstract Micro-expressions (MEs) are unconscious and involuntary reactions that genuinely reflect an individual’s inner emotional state, making them valuable in the fields of emotion analysis and behavior recognition. MEs are characterized by subtle changes within specific facial action units, and effective feature learning and fusion tailored to these characteristics still require in-depth research. To address this challenge, this paper proposes a novel hierarchical feature aggregation network (HFA-Net). In the local branch, the multi-scale attention (MSA) block is proposed to capture subtle facial changes and local information. The global branch introduces the retentive meet transformers (RMT) block to establish dependencies between holistic facial features and structural information. Considering that single-scale features are insufficient to fully capture the subtleties of MEs, a multi-level feature aggregation (MLFA) module is proposed to extract and fuse features from different levels across the two branches, preserving more comprehensive feature information. To enhance the representation of key features, an adaptive attention feature fusion (AAFF) module is designed to focus on the most useful and relevant feature channels. Extensive experiments conducted on the SMIC, CASME II, and SAMM benchmark databases demonstrate that the proposed HFA-Net outperforms current state-of-the-art methods. Additionally, ablation studies confirm the superior discriminative capability of HFA-Net when learning feature representations from limited ME samples. Our code is publicly available at https://github.com/tairuwu/HFA-Net .https://doi.org/10.1007/s40747-025-01804-0Micro-expression recognitionAttention mechanismMulti-scaleFeature fusion |
| spellingShingle | Meng Zhang Wenzhong Yang Liejun Wang Zhonghua Wu Danny Chen HFA-Net: hierarchical feature aggregation network for micro-expression recognition Complex & Intelligent Systems Micro-expression recognition Attention mechanism Multi-scale Feature fusion |
| title | HFA-Net: hierarchical feature aggregation network for micro-expression recognition |
| title_full | HFA-Net: hierarchical feature aggregation network for micro-expression recognition |
| title_fullStr | HFA-Net: hierarchical feature aggregation network for micro-expression recognition |
| title_full_unstemmed | HFA-Net: hierarchical feature aggregation network for micro-expression recognition |
| title_short | HFA-Net: hierarchical feature aggregation network for micro-expression recognition |
| title_sort | hfa net hierarchical feature aggregation network for micro expression recognition |
| topic | Micro-expression recognition Attention mechanism Multi-scale Feature fusion |
| url | https://doi.org/10.1007/s40747-025-01804-0 |
| work_keys_str_mv | AT mengzhang hfanethierarchicalfeatureaggregationnetworkformicroexpressionrecognition AT wenzhongyang hfanethierarchicalfeatureaggregationnetworkformicroexpressionrecognition AT liejunwang hfanethierarchicalfeatureaggregationnetworkformicroexpressionrecognition AT zhonghuawu hfanethierarchicalfeatureaggregationnetworkformicroexpressionrecognition AT dannychen hfanethierarchicalfeatureaggregationnetworkformicroexpressionrecognition |