TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution
Convolutional neural network (CNN) has achieved impressive success in lightweight image super-resolution (SR) methods, yet the nature of its local operations constrains the SR performance. Recent Transformer has attracted increasing attention in lightweight SR methods owing to its remarkable global...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10707600/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850266429893378048 |
|---|---|
| author | Danlin Cai Wenwen Tan Feiyang Chen Xinchi Lou Jianbin Xiahou Daxin Zhu Detian Huang |
| author_facet | Danlin Cai Wenwen Tan Feiyang Chen Xinchi Lou Jianbin Xiahou Daxin Zhu Detian Huang |
| author_sort | Danlin Cai |
| collection | DOAJ |
| description | Convolutional neural network (CNN) has achieved impressive success in lightweight image super-resolution (SR) methods, yet the nature of its local operations constrains the SR performance. Recent Transformer has attracted increasing attention in lightweight SR methods owing to its remarkable global feature extraction capacity. However, the huge computational cost makes it challenging for lightweight SR methods to efficiently utilize Transformer to exploit global contextual information from shallow to intermediate layers. To address these issues, we propose a novel lightweight Transformer and CNN interaction network for image Super-Resolution (TCSR), which fully leverages the complementary strengths of Transformer and CNN. Specifically, an efficient lightweight Transformer and CNN Interaction Block (TCIB) is designed to extract local and global features at various stages of the network, resulting in favorable hybrid features that significantly improve the quality of reconstructed images. Then, we construct a lightweight Reversed UNet (RUNet) to progressively aggregate hybrid features as well as to better trade-off the reconstruction accuracy and efficiency. Furthermore, we introduce a Refinement module to further refine edge and texture details with global information. Experimental results on numerous benchmarks validate that the proposed TCSR achieves superior performance with fewer parameters and less computational overhead than state-of-the-art lightweight methods. |
| format | Article |
| id | doaj-art-4380ee689eae43a0804e6dffcd6a168a |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-4380ee689eae43a0804e6dffcd6a168a2025-08-20T01:54:11ZengIEEEIEEE Access2169-35362024-01-011217478217479510.1109/ACCESS.2024.347636910707600TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-ResolutionDanlin Cai0https://orcid.org/0000-0003-3083-5503Wenwen Tan1Feiyang Chen2Xinchi Lou3Jianbin Xiahou4Daxin Zhu5https://orcid.org/0000-0002-8060-7368Detian Huang6https://orcid.org/0000-0002-8542-3728School of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou, ChinaCollege of Engineering, Huaqiao University, Quanzhou, ChinaCollege of Engineering, Huaqiao University, Quanzhou, ChinaSchool of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou, ChinaSchool of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou, ChinaSchool of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou, ChinaCollege of Engineering, Huaqiao University, Quanzhou, ChinaConvolutional neural network (CNN) has achieved impressive success in lightweight image super-resolution (SR) methods, yet the nature of its local operations constrains the SR performance. Recent Transformer has attracted increasing attention in lightweight SR methods owing to its remarkable global feature extraction capacity. However, the huge computational cost makes it challenging for lightweight SR methods to efficiently utilize Transformer to exploit global contextual information from shallow to intermediate layers. To address these issues, we propose a novel lightweight Transformer and CNN interaction network for image Super-Resolution (TCSR), which fully leverages the complementary strengths of Transformer and CNN. Specifically, an efficient lightweight Transformer and CNN Interaction Block (TCIB) is designed to extract local and global features at various stages of the network, resulting in favorable hybrid features that significantly improve the quality of reconstructed images. Then, we construct a lightweight Reversed UNet (RUNet) to progressively aggregate hybrid features as well as to better trade-off the reconstruction accuracy and efficiency. Furthermore, we introduce a Refinement module to further refine edge and texture details with global information. Experimental results on numerous benchmarks validate that the proposed TCSR achieves superior performance with fewer parameters and less computational overhead than state-of-the-art lightweight methods.https://ieeexplore.ieee.org/document/10707600/Lightweight image super-resolutiontransformerconvolutional neural networkUNet |
| spellingShingle | Danlin Cai Wenwen Tan Feiyang Chen Xinchi Lou Jianbin Xiahou Daxin Zhu Detian Huang TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution IEEE Access Lightweight image super-resolution transformer convolutional neural network UNet |
| title | TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution |
| title_full | TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution |
| title_fullStr | TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution |
| title_full_unstemmed | TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution |
| title_short | TCSR: Lightweight Transformer and CNN Interaction Network for Image Super-Resolution |
| title_sort | tcsr lightweight transformer and cnn interaction network for image super resolution |
| topic | Lightweight image super-resolution transformer convolutional neural network UNet |
| url | https://ieeexplore.ieee.org/document/10707600/ |
| work_keys_str_mv | AT danlincai tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT wenwentan tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT feiyangchen tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT xinchilou tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT jianbinxiahou tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT daxinzhu tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution AT detianhuang tcsrlightweighttransformerandcnninteractionnetworkforimagesuperresolution |