YOLO-Act: Unified Spatiotemporal Detection of Human Actions Across Multi-Frame Sequences

YOLO-Act: Unified Spatiotemporal Detection of Human Actions Across Multi-Frame Sequences

Automated action recognition has become essential in the surveillance, healthcare, and multimedia retrieval industries owing to the rapid proliferation of video data. This paper introduces YOLO-Act, a novel spatiotemporal action detection model that enhances the object detection capabilities of YOLO...

Full description

Saved in:
Bibliographic Details
Main Authors: Nada Alzahrani, Ouiem Bchir, Mohamed Maher Ben Ismail
Format: Article
Language:English
Published: MDPI AG 2025-05-01
Series:Sensors
Subjects:
action detection
keyframe extraction
fusion technique
spatiotemporal information
you only look once (YOLO)
Online Access:https://www.mdpi.com/1424-8220/25/10/3013
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automated action recognition has become essential in the surveillance, healthcare, and multimedia retrieval industries owing to the rapid proliferation of video data. This paper introduces YOLO-Act, a novel spatiotemporal action detection model that enhances the object detection capabilities of YOLOv8 to efficiently manage complex action dynamics within video sequences. YOLO-Act achieves precise and efficient action recognition by integrating keyframe extraction, action tracking, and class fusion. The model depicts essential temporal dynamics without the computational overhead of continuous frame processing by leveraging the adaptive selection of three keyframes representing the beginning, middle, and end of the actions. Compared with state-of-the-art approaches such as the Lagrangian Action Recognition Transformer (LART), YOLO-Act exhibits superior performance with a mean average precision (mAP) of 73.28 in experiments conducted on the AVA dataset, resulting in a gain of +28.18 mAP. Furthermore, YOLO-Act achieves this higher accuracy with significantly lower FLOPs, demonstrating its efficiency in computational resource utilization. The results highlight the advantages of incorporating precise tracking, effective spatial detection, and temporal consistency to address the challenges associated with video-based action detection.
ISSN:1424-8220

Similar Items