Reasoning action-centric temporal relations at rich feature hierarchies for action recognition.
Reasoning temporal relations among action-related objects plays an important role in action recognition. However, previous approaches only focus the reasoning on high-level semantics and inevitably involve the background in reasoning. In this work, we propose to formulate the temporal relational rea...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0327302 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849387953115103232 |
|---|---|
| author | Manshu Liang Wenbin Wu Zhuolei Chen Tengfei Han Yuan Zheng |
| author_facet | Manshu Liang Wenbin Wu Zhuolei Chen Tengfei Han Yuan Zheng |
| author_sort | Manshu Liang |
| collection | DOAJ |
| description | Reasoning temporal relations among action-related objects plays an important role in action recognition. However, previous approaches only focus the reasoning on high-level semantics and inevitably involve the background in reasoning. In this work, we propose to formulate the temporal relational reasoning in an action-centric and hierarchical style, with a novel Action-centric Temporal-relational Reasoning (ATR) block. Specifically, ATR comprises two components: an Action-related Region Locator (ARL) to locate the action-related regions via estimating the actionness, and an Efficient Action-centric Reasoner (EAR) to efficiently reason the temporal relations between the located regions so as to realize the action-centric reasoning. Thanks to its flexible and efficient designs, our ATR can be directly integrated into existing action recognition models at different depths, empowering the hierarchical reasoning on the action-centric temporal relations at the cost of minor computational overhead. We extensively evaluate our ATR block on three action recognition benchmarks, Something-Something V1, V2, and Kinetics, with the backbones of TSN, TSM, and SlowOnly. The consistent and notable improvements over the strong baselines sufficiently corroborate the effectiveness of ATR, along with the action-centric and hierarchical formulation for temporal relational reasoning. Our proposed approach provides potential practical significance for real-world scenarios. |
| format | Article |
| id | doaj-art-e41e4ec6f3cf4359a63863ce9cba7f9e |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-e41e4ec6f3cf4359a63863ce9cba7f9e2025-08-20T03:42:26ZengPublic Library of Science (PLoS)PLoS ONE1932-62032025-01-01207e032730210.1371/journal.pone.0327302Reasoning action-centric temporal relations at rich feature hierarchies for action recognition.Manshu LiangWenbin WuZhuolei ChenTengfei HanYuan ZhengReasoning temporal relations among action-related objects plays an important role in action recognition. However, previous approaches only focus the reasoning on high-level semantics and inevitably involve the background in reasoning. In this work, we propose to formulate the temporal relational reasoning in an action-centric and hierarchical style, with a novel Action-centric Temporal-relational Reasoning (ATR) block. Specifically, ATR comprises two components: an Action-related Region Locator (ARL) to locate the action-related regions via estimating the actionness, and an Efficient Action-centric Reasoner (EAR) to efficiently reason the temporal relations between the located regions so as to realize the action-centric reasoning. Thanks to its flexible and efficient designs, our ATR can be directly integrated into existing action recognition models at different depths, empowering the hierarchical reasoning on the action-centric temporal relations at the cost of minor computational overhead. We extensively evaluate our ATR block on three action recognition benchmarks, Something-Something V1, V2, and Kinetics, with the backbones of TSN, TSM, and SlowOnly. The consistent and notable improvements over the strong baselines sufficiently corroborate the effectiveness of ATR, along with the action-centric and hierarchical formulation for temporal relational reasoning. Our proposed approach provides potential practical significance for real-world scenarios.https://doi.org/10.1371/journal.pone.0327302 |
| spellingShingle | Manshu Liang Wenbin Wu Zhuolei Chen Tengfei Han Yuan Zheng Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. PLoS ONE |
| title | Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. |
| title_full | Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. |
| title_fullStr | Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. |
| title_full_unstemmed | Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. |
| title_short | Reasoning action-centric temporal relations at rich feature hierarchies for action recognition. |
| title_sort | reasoning action centric temporal relations at rich feature hierarchies for action recognition |
| url | https://doi.org/10.1371/journal.pone.0327302 |
| work_keys_str_mv | AT manshuliang reasoningactioncentrictemporalrelationsatrichfeaturehierarchiesforactionrecognition AT wenbinwu reasoningactioncentrictemporalrelationsatrichfeaturehierarchiesforactionrecognition AT zhuoleichen reasoningactioncentrictemporalrelationsatrichfeaturehierarchiesforactionrecognition AT tengfeihan reasoningactioncentrictemporalrelationsatrichfeaturehierarchiesforactionrecognition AT yuanzheng reasoningactioncentrictemporalrelationsatrichfeaturehierarchiesforactionrecognition |