Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning
Human abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-01-01
|
| Series: | Advances in Multimedia |
| Online Access: | http://dx.doi.org/10.1155/2024/4187991 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850166163911213056 |
|---|---|
| author | Hong-Bo Huang Yao-Lin Zheng Zhi-Ying Hu |
| author_facet | Hong-Bo Huang Yao-Lin Zheng Zhi-Ying Hu |
| author_sort | Hong-Bo Huang |
| collection | DOAJ |
| description | Human abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that leverages text descriptions associated with abnormal human action videos. Our method exploits the correlation between the text domain and the video domain in the semantic feature space and introduces a multimodal heterogeneous transfer learning framework from the text domain to the video domain. The text of the videos is used for feature encoding and knowledge extraction, and knowledge transfer and sharing are realized in the feature space, which is used to assist in the training of the abnormal action recognition model. The proposed method reduces the reliance on labeled video data, improves the performance of the abnormal human action recognition algorithm, and outperforms the popular video-based models, particularly in scenarios with sparse data. Moreover, our framework contributes to the advancement of automatic video analysis and abnormal action recognition, providing insights for the application of multimodal methods in a broader context. |
| format | Article |
| id | doaj-art-ddc756e2ad1a4b82a936f759a62373b3 |
| institution | OA Journals |
| issn | 1687-5699 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advances in Multimedia |
| spelling | doaj-art-ddc756e2ad1a4b82a936f759a62373b32025-08-20T02:21:33ZengWileyAdvances in Multimedia1687-56992024-01-01202410.1155/2024/4187991Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer LearningHong-Bo Huang0Yao-Lin Zheng1Zhi-Ying Hu2Computer SchoolComputer SchoolComputer SchoolHuman abnormal action recognition is crucial for video understanding and intelligent surveillance. However, the scarcity of labeled data for abnormal human actions often hinders the development of high-performance models. Inspired by the multimodal approach, this paper proposes a novel approach that leverages text descriptions associated with abnormal human action videos. Our method exploits the correlation between the text domain and the video domain in the semantic feature space and introduces a multimodal heterogeneous transfer learning framework from the text domain to the video domain. The text of the videos is used for feature encoding and knowledge extraction, and knowledge transfer and sharing are realized in the feature space, which is used to assist in the training of the abnormal action recognition model. The proposed method reduces the reliance on labeled video data, improves the performance of the abnormal human action recognition algorithm, and outperforms the popular video-based models, particularly in scenarios with sparse data. Moreover, our framework contributes to the advancement of automatic video analysis and abnormal action recognition, providing insights for the application of multimodal methods in a broader context.http://dx.doi.org/10.1155/2024/4187991 |
| spellingShingle | Hong-Bo Huang Yao-Lin Zheng Zhi-Ying Hu Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning Advances in Multimedia |
| title | Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning |
| title_full | Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning |
| title_fullStr | Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning |
| title_full_unstemmed | Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning |
| title_short | Video Abnormal Action Recognition Based on Multimodal Heterogeneous Transfer Learning |
| title_sort | video abnormal action recognition based on multimodal heterogeneous transfer learning |
| url | http://dx.doi.org/10.1155/2024/4187991 |
| work_keys_str_mv | AT hongbohuang videoabnormalactionrecognitionbasedonmultimodalheterogeneoustransferlearning AT yaolinzheng videoabnormalactionrecognitionbasedonmultimodalheterogeneoustransferlearning AT zhiyinghu videoabnormalactionrecognitionbasedonmultimodalheterogeneoustransferlearning |