Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection
Abstract It is important in the rising demands to have efficient anomaly detection in camera surveillance systems for improving public safety in a complex environment. Most of the available methods usually fail to capture the long-term temporal dependencies and spatial correlations, especially in dy...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-85822-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585683509706752 |
|---|---|
| author | Sai Babu Veesam Aravapalli Rama Satish Sreenivasulu Tupakula Yuvaraju Chinnam Krishna Prakash Shonak Bansal Mohammad Rashed Iqbal Faruque |
| author_facet | Sai Babu Veesam Aravapalli Rama Satish Sreenivasulu Tupakula Yuvaraju Chinnam Krishna Prakash Shonak Bansal Mohammad Rashed Iqbal Faruque |
| author_sort | Sai Babu Veesam |
| collection | DOAJ |
| description | Abstract It is important in the rising demands to have efficient anomaly detection in camera surveillance systems for improving public safety in a complex environment. Most of the available methods usually fail to capture the long-term temporal dependencies and spatial correlations, especially in dynamic multi-camera settings. Also, many traditional methods rely heavily on large labeled datasets, generalizing poorly when encountering unseen anomalies in the process. We introduce a new framework to address such challenges by incorporating state-of-the-art deep learning models that improve temporal and spatial context modeling. We combine RNNs with GATs to model long-term dependencies across cameras effectively distributed over space. The Transformer-Augmented RNN allows for a better way than standard RNNs through self-attention mechanisms to improve robust temporal modeling. We employ a Multimodal Variational Autoencoder-MVAE that fuses video, audio, and motion sensor information in a manner resistant to noise and missing samples. To address the challenge of having a few labeled anomalies, we apply the Prototypical Networks to perform few-shot learning and enable generalization based on a few examples. Then, a Spatiotemporal Autoencoder is adopted to realize unsupervised anomaly detection by learning normal behavior patterns and deviations from them as anomalies. The methods proposed here yield significant improvements of about 10% to 15% in precision, recall, and F1-scores over traditional models. Further, the generalization capability of the framework to unseen anomalies, up to a gain of + 20% on novel event detection, represents a major advancement for real-world surveillance systems. |
| format | Article |
| id | doaj-art-0954db36c8074bd7ae196d5115c9b9bb |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-0954db36c8074bd7ae196d5115c9b9bb2025-01-26T12:34:28ZengNature PortfolioScientific Reports2045-23222025-01-0115112410.1038/s41598-025-85822-5Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detectionSai Babu Veesam0Aravapalli Rama Satish1Sreenivasulu Tupakula2Yuvaraju Chinnam3Krishna Prakash4Shonak Bansal5Mohammad Rashed Iqbal Faruque6School of Computer Science, VIT-AP UniversitySchool of Computer Science, VIT-AP UniversityDepartment of Electronics and Communication Engineering, SRM UniversityProfessor of Computer Science and Engineering (AI&ML), St. Peter’s Engineering CollegeDepartment of Electronics and Communication Engineering, NRI Institute of TechnologyDepartment of Electronics and Communication Engineering, University Institute of Engineering, Chandigarh UniversitySpace Science Centre (ANGKASA), Universiti Kebangsaan MalaysiaAbstract It is important in the rising demands to have efficient anomaly detection in camera surveillance systems for improving public safety in a complex environment. Most of the available methods usually fail to capture the long-term temporal dependencies and spatial correlations, especially in dynamic multi-camera settings. Also, many traditional methods rely heavily on large labeled datasets, generalizing poorly when encountering unseen anomalies in the process. We introduce a new framework to address such challenges by incorporating state-of-the-art deep learning models that improve temporal and spatial context modeling. We combine RNNs with GATs to model long-term dependencies across cameras effectively distributed over space. The Transformer-Augmented RNN allows for a better way than standard RNNs through self-attention mechanisms to improve robust temporal modeling. We employ a Multimodal Variational Autoencoder-MVAE that fuses video, audio, and motion sensor information in a manner resistant to noise and missing samples. To address the challenge of having a few labeled anomalies, we apply the Prototypical Networks to perform few-shot learning and enable generalization based on a few examples. Then, a Spatiotemporal Autoencoder is adopted to realize unsupervised anomaly detection by learning normal behavior patterns and deviations from them as anomalies. The methods proposed here yield significant improvements of about 10% to 15% in precision, recall, and F1-scores over traditional models. Further, the generalization capability of the framework to unseen anomalies, up to a gain of + 20% on novel event detection, represents a major advancement for real-world surveillance systems.https://doi.org/10.1038/s41598-025-85822-5Anomaly detectionTemporal graph attentionTransformer-augmented RNNsMultimodal fusionFew-shot learning |
| spellingShingle | Sai Babu Veesam Aravapalli Rama Satish Sreenivasulu Tupakula Yuvaraju Chinnam Krishna Prakash Shonak Bansal Mohammad Rashed Iqbal Faruque Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection Scientific Reports Anomaly detection Temporal graph attention Transformer-augmented RNNs Multimodal fusion Few-shot learning |
| title | Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection |
| title_full | Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection |
| title_fullStr | Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection |
| title_full_unstemmed | Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection |
| title_short | Design of an integrated model with temporal graph attention and transformer-augmented RNNs for enhanced anomaly detection |
| title_sort | design of an integrated model with temporal graph attention and transformer augmented rnns for enhanced anomaly detection |
| topic | Anomaly detection Temporal graph attention Transformer-augmented RNNs Multimodal fusion Few-shot learning |
| url | https://doi.org/10.1038/s41598-025-85822-5 |
| work_keys_str_mv | AT saibabuveesam designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT aravapalliramasatish designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT sreenivasulutupakula designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT yuvarajuchinnam designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT krishnaprakash designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT shonakbansal designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection AT mohammadrashediqbalfaruque designofanintegratedmodelwithtemporalgraphattentionandtransformeraugmentedrnnsforenhancedanomalydetection |