A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration
Image restoration under adverse weather conditions is of great significance for the subsequent advanced computer vision tasks. However, most existing image restoration algorithms only remove single weather degradation, and few studies has been conducted on all-in-one weather-degraded image restorati...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Office of Journal of Shanghai Jiao Tong University
2024-10-01
|
| Series: | Shanghai Jiaotong Daxue xuebao |
| Subjects: | |
| Online Access: | https://xuebao.sjtu.edu.cn/article/2024/1006-2467/1006-2467-58-10-1606.shtml |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850284904868216832 |
|---|---|
| author | QIN Jing, WEN Yuanbo, GAO Tao, LIU Yao |
| author_facet | QIN Jing, WEN Yuanbo, GAO Tao, LIU Yao |
| author_sort | QIN Jing, WEN Yuanbo, GAO Tao, LIU Yao |
| collection | DOAJ |
| description | Image restoration under adverse weather conditions is of great significance for the subsequent advanced computer vision tasks. However, most existing image restoration algorithms only remove single weather degradation, and few studies has been conducted on all-in-one weather-degraded image restoration. The denoising diffusion probability model is combined with Vision Transformer to propose a Transformer-based diffusion model for all-in-one weather-degraded image restoration. First, the weather-degraded image is utilized as the condition to guide the reverse sampling of diffusion model and generate corresponding clean background image. Then, the subspace transposed Transformer for noise estimation (NE-STT) is proposed, which utilizes the degraded image and the noisy state to estimate noise distribution, including the subspace transposed self-attention (STSA) mechanism and a dual grouped gated feed-forward network (DGGFFN). The STSA adopts subspace transformation coefficient to effectively capture global long-range dependencies while significantly reducing computational burden. The DGGFFN employs the dual grouped gated mechanism to enhance the nonlinear characterization ability of feed-forward network. The experimental results show that in comparison with the recently developed algorithms, such as All-in-One and TransWeather, the method proposed obtains a performance gain of 3.68 and 3.08 dB in average peak signal-to-noise ratio while 2.93% and 3.13% in average structural similarity on 5 weather-degraded datasets. |
| format | Article |
| id | doaj-art-9896f0da2f7a4115afe58cca647f8d67 |
| institution | OA Journals |
| issn | 1006-2467 |
| language | zho |
| publishDate | 2024-10-01 |
| publisher | Editorial Office of Journal of Shanghai Jiao Tong University |
| record_format | Article |
| series | Shanghai Jiaotong Daxue xuebao |
| spelling | doaj-art-9896f0da2f7a4115afe58cca647f8d672025-08-20T01:47:26ZzhoEditorial Office of Journal of Shanghai Jiao Tong UniversityShanghai Jiaotong Daxue xuebao1006-24672024-10-0158101606161710.16183/j.cnki.jsjtu.2023.043A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image RestorationQIN Jing, WEN Yuanbo, GAO Tao, LIU Yao0School of Information and Engineering, Chang’an University, Xi’an 710064, ChinaImage restoration under adverse weather conditions is of great significance for the subsequent advanced computer vision tasks. However, most existing image restoration algorithms only remove single weather degradation, and few studies has been conducted on all-in-one weather-degraded image restoration. The denoising diffusion probability model is combined with Vision Transformer to propose a Transformer-based diffusion model for all-in-one weather-degraded image restoration. First, the weather-degraded image is utilized as the condition to guide the reverse sampling of diffusion model and generate corresponding clean background image. Then, the subspace transposed Transformer for noise estimation (NE-STT) is proposed, which utilizes the degraded image and the noisy state to estimate noise distribution, including the subspace transposed self-attention (STSA) mechanism and a dual grouped gated feed-forward network (DGGFFN). The STSA adopts subspace transformation coefficient to effectively capture global long-range dependencies while significantly reducing computational burden. The DGGFFN employs the dual grouped gated mechanism to enhance the nonlinear characterization ability of feed-forward network. The experimental results show that in comparison with the recently developed algorithms, such as All-in-One and TransWeather, the method proposed obtains a performance gain of 3.68 and 3.08 dB in average peak signal-to-noise ratio while 2.93% and 3.13% in average structural similarity on 5 weather-degraded datasets.https://xuebao.sjtu.edu.cn/article/2024/1006-2467/1006-2467-58-10-1606.shtmlcomputer visiondiffusion modelimage restorationtransformerweather-degraded image |
| spellingShingle | QIN Jing, WEN Yuanbo, GAO Tao, LIU Yao A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration Shanghai Jiaotong Daxue xuebao computer vision diffusion model image restoration transformer weather-degraded image |
| title | A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration |
| title_full | A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration |
| title_fullStr | A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration |
| title_full_unstemmed | A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration |
| title_short | A Transformer-Based Diffusion Model for All-in-One Weather-Degraded Image Restoration |
| title_sort | transformer based diffusion model for all in one weather degraded image restoration |
| topic | computer vision diffusion model image restoration transformer weather-degraded image |
| url | https://xuebao.sjtu.edu.cn/article/2024/1006-2467/1006-2467-58-10-1606.shtml |
| work_keys_str_mv | AT qinjingwenyuanbogaotaoliuyao atransformerbaseddiffusionmodelforallinoneweatherdegradedimagerestoration AT qinjingwenyuanbogaotaoliuyao transformerbaseddiffusionmodelforallinoneweatherdegradedimagerestoration |