Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction
Recently, STDenseNet (SpatioTemporal Densely connected convolutional Network) showed remarkable performance in predicting network traffic by leveraging the inductive bias of convolution layers. However, it is known that such convolution layers can only barely capture long-term spatial and temporal d...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Signals |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2624-6120/6/2/29 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849467269046861824 |
|---|---|
| author | Myeongjun Oh Sung Oh Jongkyung Im Myungho Kim Joung-Sik Kim Ji-Yeon Park Na-Rae Yi Sung-Ho Bae |
| author_facet | Myeongjun Oh Sung Oh Jongkyung Im Myungho Kim Joung-Sik Kim Ji-Yeon Park Na-Rae Yi Sung-Ho Bae |
| author_sort | Myeongjun Oh |
| collection | DOAJ |
| description | Recently, STDenseNet (SpatioTemporal Densely connected convolutional Network) showed remarkable performance in predicting network traffic by leveraging the inductive bias of convolution layers. However, it is known that such convolution layers can only barely capture long-term spatial and temporal dependencies. To solve this problem, we propose Attention-DenseNet (ADNet), which effectively incorporates an attention module into STDenseNet to learn representations for long-term spatio-temporal patterns. Specifically, we explored the optimal positions and the types of attention modules in combination with STDenseNet. Our key findings are as follows: i) attention modules are very effective when positioned between the last dense module and the final feature fusion module, meaning that the attention module plays a key role in aggregating low-level local features with long-term dependency. Hence, the final feature fusion module can easily exploit both global and local information; ii) the best attention module is different depending on the spatio-temporal characteristics of the dataset. To verify the effectiveness of the proposed ADNet, we performed experiments on the Telecom Italia dataset, a well-known benchmark dataset for network traffic prediction. The experimental results show that, compared to STDenseNet, our ADNet improved RMSE performance by 3.72%, 2.84%, and 5.87% in call service (Call), short message service (SMS), and Internet access (Internet) sub-datasets, respectively. |
| format | Article |
| id | doaj-art-7a164ecc9aa84f83aca39ecad1454bcf |
| institution | Kabale University |
| issn | 2624-6120 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Signals |
| spelling | doaj-art-7a164ecc9aa84f83aca39ecad1454bcf2025-08-20T03:27:43ZengMDPI AGSignals2624-61202025-06-01622910.3390/signals6020029Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic PredictionMyeongjun Oh0Sung Oh1Jongkyung Im2Myungho Kim3Joung-Sik Kim4Ji-Yeon Park5Na-Rae Yi6Sung-Ho Bae7Department of Artificial Intelligence, Kyunghee University, 1732, Deogyeong-daero, Giheung-gu, Yongin-si 17104, Republic of KoreaDepartment of Artificial Intelligence, Kyunghee University, 1732, Deogyeong-daero, Giheung-gu, Yongin-si 17104, Republic of KoreaDepartment of Artificial Intelligence, Kyunghee University, 1732, Deogyeong-daero, Giheung-gu, Yongin-si 17104, Republic of KoreaHanwha, 188 Pangyoyeok-ro, Bundang-gu, Seongnam-si 13524, Republic of KoreaHanwha, 188 Pangyoyeok-ro, Bundang-gu, Seongnam-si 13524, Republic of KoreaHanwha, 188 Pangyoyeok-ro, Bundang-gu, Seongnam-si 13524, Republic of KoreaHanwha, 188 Pangyoyeok-ro, Bundang-gu, Seongnam-si 13524, Republic of KoreaDepartment of Artificial Intelligence, Kyunghee University, 1732, Deogyeong-daero, Giheung-gu, Yongin-si 17104, Republic of KoreaRecently, STDenseNet (SpatioTemporal Densely connected convolutional Network) showed remarkable performance in predicting network traffic by leveraging the inductive bias of convolution layers. However, it is known that such convolution layers can only barely capture long-term spatial and temporal dependencies. To solve this problem, we propose Attention-DenseNet (ADNet), which effectively incorporates an attention module into STDenseNet to learn representations for long-term spatio-temporal patterns. Specifically, we explored the optimal positions and the types of attention modules in combination with STDenseNet. Our key findings are as follows: i) attention modules are very effective when positioned between the last dense module and the final feature fusion module, meaning that the attention module plays a key role in aggregating low-level local features with long-term dependency. Hence, the final feature fusion module can easily exploit both global and local information; ii) the best attention module is different depending on the spatio-temporal characteristics of the dataset. To verify the effectiveness of the proposed ADNet, we performed experiments on the Telecom Italia dataset, a well-known benchmark dataset for network traffic prediction. The experimental results show that, compared to STDenseNet, our ADNet improved RMSE performance by 3.72%, 2.84%, and 5.87% in call service (Call), short message service (SMS), and Internet access (Internet) sub-datasets, respectively.https://www.mdpi.com/2624-6120/6/2/29spatiotemporal data predictionDenseNetattention mechanism |
| spellingShingle | Myeongjun Oh Sung Oh Jongkyung Im Myungho Kim Joung-Sik Kim Ji-Yeon Park Na-Rae Yi Sung-Ho Bae Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction Signals spatiotemporal data prediction DenseNet attention mechanism |
| title | Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction |
| title_full | Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction |
| title_fullStr | Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction |
| title_full_unstemmed | Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction |
| title_short | Comparative Analysis of Attention Mechanisms in Densely Connected Network for Network Traffic Prediction |
| title_sort | comparative analysis of attention mechanisms in densely connected network for network traffic prediction |
| topic | spatiotemporal data prediction DenseNet attention mechanism |
| url | https://www.mdpi.com/2624-6120/6/2/29 |
| work_keys_str_mv | AT myeongjunoh comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT sungoh comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT jongkyungim comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT myunghokim comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT joungsikkim comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT jiyeonpark comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT naraeyi comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction AT sunghobae comparativeanalysisofattentionmechanismsindenselyconnectednetworkfornetworktrafficprediction |