SVTSR: image super-resolution using scattering vision transformer
Abstract Vision transformers have garnered substantial attention and attained impressive performance in image super-resolution tasks. Nevertheless, these networks face challenges associated with attention complexity and the effective capture of intricate, fine-grained details within images. These hu...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-82650-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559489719828480 |
|---|---|
| author | Jiabao Liang Yutao Jin Xiaoyan Chen Haotian Huang Yue Deng |
| author_facet | Jiabao Liang Yutao Jin Xiaoyan Chen Haotian Huang Yue Deng |
| author_sort | Jiabao Liang |
| collection | DOAJ |
| description | Abstract Vision transformers have garnered substantial attention and attained impressive performance in image super-resolution tasks. Nevertheless, these networks face challenges associated with attention complexity and the effective capture of intricate, fine-grained details within images. These hurdles impede the efficient and scalable deployment of transformer models for image super-resolution tasks in real-world applications. In this paper, we present a novel vision transformer called Scattering Vision Transformer for Super-Resolution (SVTSR) to tackle these challenges. SVTSR integrates a spectrally scattering network to efficiently capture intricate image details. It addresses the invertibility problem commonly encountered in down-sampling operations by separating low-frequency and high-frequency components. Additionally, SVTSR introduces a novel spectral gating network that utilizes Einstein multiplication for token and channel mixing, effectively reducing complexity. Extensive experiments show the effectiveness of the proposed vision transformer for image super-resolution tasks. Our comprehensive methodology not only outperforms state-of-the-art methods in terms of the PSNR and SSIM metrics but, more significantly, entails a reduction in model parameters exceeding tenfold when compared to the baseline model. As shown in Fig. 1, the substantial decrease of parameter amount proves highly advantageous for the deployment and practical application of super-resolution models. Code is available at https://github.com/LiangJiabaoY/SVTSR.git. |
| format | Article |
| id | doaj-art-30e186b66cb84d2981bfffccaf9180e2 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-30e186b66cb84d2981bfffccaf9180e22025-01-05T12:25:22ZengNature PortfolioScientific Reports2045-23222024-12-0114111110.1038/s41598-024-82650-xSVTSR: image super-resolution using scattering vision transformerJiabao Liang0Yutao Jin1Xiaoyan Chen2Haotian Huang3Yue Deng4School of Electronic Information and AutomationSchool of Electronic Information and AutomationSchool of Electronic Information and AutomationSchool of Electronic Information and AutomationSchool of Electronic Information and AutomationAbstract Vision transformers have garnered substantial attention and attained impressive performance in image super-resolution tasks. Nevertheless, these networks face challenges associated with attention complexity and the effective capture of intricate, fine-grained details within images. These hurdles impede the efficient and scalable deployment of transformer models for image super-resolution tasks in real-world applications. In this paper, we present a novel vision transformer called Scattering Vision Transformer for Super-Resolution (SVTSR) to tackle these challenges. SVTSR integrates a spectrally scattering network to efficiently capture intricate image details. It addresses the invertibility problem commonly encountered in down-sampling operations by separating low-frequency and high-frequency components. Additionally, SVTSR introduces a novel spectral gating network that utilizes Einstein multiplication for token and channel mixing, effectively reducing complexity. Extensive experiments show the effectiveness of the proposed vision transformer for image super-resolution tasks. Our comprehensive methodology not only outperforms state-of-the-art methods in terms of the PSNR and SSIM metrics but, more significantly, entails a reduction in model parameters exceeding tenfold when compared to the baseline model. As shown in Fig. 1, the substantial decrease of parameter amount proves highly advantageous for the deployment and practical application of super-resolution models. Code is available at https://github.com/LiangJiabaoY/SVTSR.git.https://doi.org/10.1038/s41598-024-82650-xDual-time complex wavelet transformsEinstein blending methodTensor blending method |
| spellingShingle | Jiabao Liang Yutao Jin Xiaoyan Chen Haotian Huang Yue Deng SVTSR: image super-resolution using scattering vision transformer Scientific Reports Dual-time complex wavelet transforms Einstein blending method Tensor blending method |
| title | SVTSR: image super-resolution using scattering vision transformer |
| title_full | SVTSR: image super-resolution using scattering vision transformer |
| title_fullStr | SVTSR: image super-resolution using scattering vision transformer |
| title_full_unstemmed | SVTSR: image super-resolution using scattering vision transformer |
| title_short | SVTSR: image super-resolution using scattering vision transformer |
| title_sort | svtsr image super resolution using scattering vision transformer |
| topic | Dual-time complex wavelet transforms Einstein blending method Tensor blending method |
| url | https://doi.org/10.1038/s41598-024-82650-x |
| work_keys_str_mv | AT jiabaoliang svtsrimagesuperresolutionusingscatteringvisiontransformer AT yutaojin svtsrimagesuperresolutionusingscatteringvisiontransformer AT xiaoyanchen svtsrimagesuperresolutionusingscatteringvisiontransformer AT haotianhuang svtsrimagesuperresolutionusingscatteringvisiontransformer AT yuedeng svtsrimagesuperresolutionusingscatteringvisiontransformer |