Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning
This paper presents a real-time system for the detection and classification of facial micro-expressions, evaluated on the CASME II dataset. Micro-expressions are brief and subtle indicators of genuine emotions, posing significant challenges for automatic recognition due to their low intensity, short...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/12/6417 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849434036671348736 |
|---|---|
| author | Gabriela Laura Sălăgean Monica Leba Andreea Cristina Ionica |
| author_facet | Gabriela Laura Sălăgean Monica Leba Andreea Cristina Ionica |
| author_sort | Gabriela Laura Sălăgean |
| collection | DOAJ |
| description | This paper presents a real-time system for the detection and classification of facial micro-expressions, evaluated on the CASME II dataset. Micro-expressions are brief and subtle indicators of genuine emotions, posing significant challenges for automatic recognition due to their low intensity, short duration, and inter-subject variability. To address these challenges, the proposed system integrates advanced computer vision techniques, rule-based classification grounded in the Facial Action Coding System, and artificial intelligence components. The architecture employs MediaPipe for facial landmark tracking and action unit extraction, expert rules to resolve common emotional confusions, and deep learning modules for optimized classification. Experimental validation demonstrated a classification accuracy of 93.30% on CASME II, highlighting the effectiveness of the hybrid design. The system also incorporates mechanisms for amplifying weak signals and adapting to new subjects through continuous knowledge updates. These results confirm the advantages of combining domain expertise with AI-driven reasoning to improve micro-expression recognition. The proposed methodology has practical implications for various fields, including clinical psychology, security, marketing, and human-computer interaction, where the accurate interpretation of emotional micro-signals is essential. |
| format | Article |
| id | doaj-art-306f47c505f34ac190299d05d16d8b8a |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-306f47c505f34ac190299d05d16d8b8a2025-08-20T03:26:48ZengMDPI AGApplied Sciences2076-34172025-06-011512641710.3390/app15126417Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based ReasoningGabriela Laura Sălăgean0Monica Leba1Andreea Cristina Ionica2Doctoral School, University of Petroșani, 332006 Petrosani, RomaniaSystem Control and Computer Engineering Department, University of Petroșani, 332006 Petrosani, RomaniaManagement and Industrial Engineering Department, University of Petroșani, 332006 Petrosani, RomaniaThis paper presents a real-time system for the detection and classification of facial micro-expressions, evaluated on the CASME II dataset. Micro-expressions are brief and subtle indicators of genuine emotions, posing significant challenges for automatic recognition due to their low intensity, short duration, and inter-subject variability. To address these challenges, the proposed system integrates advanced computer vision techniques, rule-based classification grounded in the Facial Action Coding System, and artificial intelligence components. The architecture employs MediaPipe for facial landmark tracking and action unit extraction, expert rules to resolve common emotional confusions, and deep learning modules for optimized classification. Experimental validation demonstrated a classification accuracy of 93.30% on CASME II, highlighting the effectiveness of the hybrid design. The system also incorporates mechanisms for amplifying weak signals and adapting to new subjects through continuous knowledge updates. These results confirm the advantages of combining domain expertise with AI-driven reasoning to improve micro-expression recognition. The proposed methodology has practical implications for various fields, including clinical psychology, security, marketing, and human-computer interaction, where the accurate interpretation of emotional micro-signals is essential.https://www.mdpi.com/2076-3417/15/12/6417affective computingfacial action coding systememotion classificationhuman-computer interactionmultimodal AI |
| spellingShingle | Gabriela Laura Sălăgean Monica Leba Andreea Cristina Ionica Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning Applied Sciences affective computing facial action coding system emotion classification human-computer interaction multimodal AI |
| title | Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning |
| title_full | Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning |
| title_fullStr | Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning |
| title_full_unstemmed | Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning |
| title_short | Seeing the Unseen: Real-Time Micro-Expression Recognition with Action Units and GPT-Based Reasoning |
| title_sort | seeing the unseen real time micro expression recognition with action units and gpt based reasoning |
| topic | affective computing facial action coding system emotion classification human-computer interaction multimodal AI |
| url | https://www.mdpi.com/2076-3417/15/12/6417 |
| work_keys_str_mv | AT gabrielalaurasalagean seeingtheunseenrealtimemicroexpressionrecognitionwithactionunitsandgptbasedreasoning AT monicaleba seeingtheunseenrealtimemicroexpressionrecognitionwithactionunitsandgptbasedreasoning AT andreeacristinaionica seeingtheunseenrealtimemicroexpressionrecognitionwithactionunitsandgptbasedreasoning |