Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism
In this work, we present a novel unsupervised scheme named SegSum, designed for video summarization through the creation of video skims. Most contemporary methods involve training a summarizer to assign importance scores to individual video frames, which are then aggregated to calculate scores for v...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10904447/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850023617600946176 |
|---|---|
| author | Vo Quoc Bang Vo Hoai Viet |
| author_facet | Vo Quoc Bang Vo Hoai Viet |
| author_sort | Vo Quoc Bang |
| collection | DOAJ |
| description | In this work, we present a novel unsupervised scheme named SegSum, designed for video summarization through the creation of video skims. Most contemporary methods involve training a summarizer to assign importance scores to individual video frames, which are then aggregated to calculate scores for video segments produced by methods like Kernel Temporal Segmentation(KTS). Nonetheless, this methodology restricts the summarizer’s access to vital information essential for generating the summary—specifically, spatial-temporal relationships in video segments. Our proposed method incorporates the segment information obtained from KTS into the learning process of the summarizer based on concentrated attention architecture in deep learning models. In our experiment, we extensively evaluated our method across several datasets and many architectural frameworks for unsupervised video summarization. By incorporating a concentrated attention module, we managed to secure top F1-scores on established benchmarks, recording 54% on the SumMe dataset and 62% on the TVSum dataset. Furthermore, even with a straightforward Regressor network, SegSum demonstrates competitive performance, producing summaries that closely align with human annotations. |
| format | Article |
| id | doaj-art-28cb6fa7c0dd4a9db95f2dd8dd343402 |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-28cb6fa7c0dd4a9db95f2dd8dd3434022025-08-20T03:01:19ZengIEEEIEEE Access2169-35362025-01-0113381473816210.1109/ACCESS.2025.354614910904447Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention MechanismVo Quoc Bang0https://orcid.org/0009-0003-0888-6504Vo Hoai Viet1https://orcid.org/0009-0002-7943-8621Faculty of Information Technology, University of Science, VNU-HCMC, Ho Chi Minh City, VietnamFaculty of Information Technology, University of Science, VNU-HCMC, Ho Chi Minh City, VietnamIn this work, we present a novel unsupervised scheme named SegSum, designed for video summarization through the creation of video skims. Most contemporary methods involve training a summarizer to assign importance scores to individual video frames, which are then aggregated to calculate scores for video segments produced by methods like Kernel Temporal Segmentation(KTS). Nonetheless, this methodology restricts the summarizer’s access to vital information essential for generating the summary—specifically, spatial-temporal relationships in video segments. Our proposed method incorporates the segment information obtained from KTS into the learning process of the summarizer based on concentrated attention architecture in deep learning models. In our experiment, we extensively evaluated our method across several datasets and many architectural frameworks for unsupervised video summarization. By incorporating a concentrated attention module, we managed to secure top F1-scores on established benchmarks, recording 54% on the SumMe dataset and 62% on the TVSum dataset. Furthermore, even with a straightforward Regressor network, SegSum demonstrates competitive performance, producing summaries that closely align with human annotations.https://ieeexplore.ieee.org/document/10904447/Video summarizationunsupervised learningtemporal video segmentation |
| spellingShingle | Vo Quoc Bang Vo Hoai Viet Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism IEEE Access Video summarization unsupervised learning temporal video segmentation |
| title | Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism |
| title_full | Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism |
| title_fullStr | Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism |
| title_full_unstemmed | Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism |
| title_short | Integrate the Temporal Scheme for Unsupervised Video Summarization via Attention Mechanism |
| title_sort | integrate the temporal scheme for unsupervised video summarization via attention mechanism |
| topic | Video summarization unsupervised learning temporal video segmentation |
| url | https://ieeexplore.ieee.org/document/10904447/ |
| work_keys_str_mv | AT voquocbang integratethetemporalschemeforunsupervisedvideosummarizationviaattentionmechanism AT vohoaiviet integratethetemporalschemeforunsupervisedvideosummarizationviaattentionmechanism |