Research on a Microexpression Recognition Technology Based on Multimodal Fusion
Microexpressions have extremely high due value in national security, public safety, medical, and other fields. However, microexpressions have characteristics that are obviously different from macroexpressions, such as short duration and weak changes, which greatly increase the difficulty of microexp...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2021/5221950 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849683748403019776 |
|---|---|
| author | Jie Kang Xiao Ying Chen Qi Yuan Liu Si Han Jin Cheng Han Yang Cong Hu |
| author_facet | Jie Kang Xiao Ying Chen Qi Yuan Liu Si Han Jin Cheng Han Yang Cong Hu |
| author_sort | Jie Kang |
| collection | DOAJ |
| description | Microexpressions have extremely high due value in national security, public safety, medical, and other fields. However, microexpressions have characteristics that are obviously different from macroexpressions, such as short duration and weak changes, which greatly increase the difficulty of microexpression recognition work. In this paper, we propose a microexpression recognition method based on multimodal fusion through a comparative study of traditional microexpression recognition algorithms such as LBP algorithm and CNN and LSTM deep learning algorithms. The method couples the separate microexpression image information with the corresponding body temperature information to establish a multimodal fusion microexpression database. This paper firstly introduces how to build a multimodal fusion microexpression database in a laboratory environment, secondly compares the recognition accuracy of LBP, LSTM, and CNN + LSTM networks for microexpressions, and finally selects the superior CNN + LSTM network in the comparison results for model training and testing on the test set under separate microexpression database and multimodal fusion database. The experimental results show that a microexpression recognition method based on multimodal fusion designed in this paper is more accurate than unimodal recognition in multimodal recognition after feature fusion, and its recognition rate reaches 75.1%, which proves that the method is feasible and effective in improving microexpression recognition rate and has good practical value. |
| format | Article |
| id | doaj-art-cec08454bf1b4ca6bc7a9c97db54510d |
| institution | DOAJ |
| issn | 1099-0526 |
| language | English |
| publishDate | 2021-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-cec08454bf1b4ca6bc7a9c97db54510d2025-08-20T03:23:43ZengWileyComplexity1099-05262021-01-01202110.1155/2021/5221950Research on a Microexpression Recognition Technology Based on Multimodal FusionJie Kang0Xiao Ying Chen1Qi Yuan Liu2Si Han Jin3Cheng Han Yang4Cong Hu5College of Mechanical & Electrical EngineeringCollege of Mechanical & Electrical EngineeringCollege of Mechanical & Electrical EngineeringCollege of Mechanical & Electrical EngineeringCollege of Mechanical & Electrical EngineeringGuangxi Key Laboratory of Automatic Detecting Technology and InstrumentsMicroexpressions have extremely high due value in national security, public safety, medical, and other fields. However, microexpressions have characteristics that are obviously different from macroexpressions, such as short duration and weak changes, which greatly increase the difficulty of microexpression recognition work. In this paper, we propose a microexpression recognition method based on multimodal fusion through a comparative study of traditional microexpression recognition algorithms such as LBP algorithm and CNN and LSTM deep learning algorithms. The method couples the separate microexpression image information with the corresponding body temperature information to establish a multimodal fusion microexpression database. This paper firstly introduces how to build a multimodal fusion microexpression database in a laboratory environment, secondly compares the recognition accuracy of LBP, LSTM, and CNN + LSTM networks for microexpressions, and finally selects the superior CNN + LSTM network in the comparison results for model training and testing on the test set under separate microexpression database and multimodal fusion database. The experimental results show that a microexpression recognition method based on multimodal fusion designed in this paper is more accurate than unimodal recognition in multimodal recognition after feature fusion, and its recognition rate reaches 75.1%, which proves that the method is feasible and effective in improving microexpression recognition rate and has good practical value.http://dx.doi.org/10.1155/2021/5221950 |
| spellingShingle | Jie Kang Xiao Ying Chen Qi Yuan Liu Si Han Jin Cheng Han Yang Cong Hu Research on a Microexpression Recognition Technology Based on Multimodal Fusion Complexity |
| title | Research on a Microexpression Recognition Technology Based on Multimodal Fusion |
| title_full | Research on a Microexpression Recognition Technology Based on Multimodal Fusion |
| title_fullStr | Research on a Microexpression Recognition Technology Based on Multimodal Fusion |
| title_full_unstemmed | Research on a Microexpression Recognition Technology Based on Multimodal Fusion |
| title_short | Research on a Microexpression Recognition Technology Based on Multimodal Fusion |
| title_sort | research on a microexpression recognition technology based on multimodal fusion |
| url | http://dx.doi.org/10.1155/2021/5221950 |
| work_keys_str_mv | AT jiekang researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion AT xiaoyingchen researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion AT qiyuanliu researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion AT sihanjin researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion AT chenghanyang researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion AT conghu researchonamicroexpressionrecognitiontechnologybasedonmultimodalfusion |