Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors
Bad weather, such as rain, snow, and fog, will reduce the quality of the image acquired, and it will affect the performance of many related visual fields. The existing researches on image restoration under severe weather either focus on the restoration under a certain kind of weather, which cannot b...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10810379/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850051877851365376 |
|---|---|
| author | Lingwen Meng Mingyong Xin Junwei Zhang Fu Zou Qingqing Zhao |
| author_facet | Lingwen Meng Mingyong Xin Junwei Zhang Fu Zou Qingqing Zhao |
| author_sort | Lingwen Meng |
| collection | DOAJ |
| description | Bad weather, such as rain, snow, and fog, will reduce the quality of the image acquired, and it will affect the performance of many related visual fields. The existing researches on image restoration under severe weather either focus on the restoration under a certain kind of weather, which cannot be generalized under different weather, or need to introduce additional model structures, which increases the burden of practical applications. Therefore, we explores an integrated image restoration framework, which can restore images under different adverse weather effects. Specifically, the model is trained in two stages. In the pre-training stage, a basic general network that can deal with multiple weather is trained by supervised learning. In the fine-tuning stage, soft prompts are introduced to stimulate the model’s ability to cope with different weather, so as to enhance the generalization ability. Specifically, the hidden information of the prompts is explored by low-rank decomposition, and the contras loss is added to prompt the prompts to converge on similar tasks. Furthermore, we address the distribution shift problem by aligning out-of-distribution (OOD) test sample statistics with those of the source data using test-time prompt tuning. Finally, we use the evaluation metrics, PSNR and SSIM, to evaluate the proposed method under four tasks: rain removal, snow removal, fog removal, and raindrop removal. The results demonstrate that the proposed method achieves superior performance compared to the existing state-of-the-art. |
| format | Article |
| id | doaj-art-eba199e3134b4ea785dbf9638ca09c87 |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-eba199e3134b4ea785dbf9638ca09c872025-08-20T02:52:59ZengIEEEIEEE Access2169-35362024-01-011219738919739910.1109/ACCESS.2024.352048410810379Multi-Task Image Restoration Algorithm Under Different Weather Influence FactorsLingwen Meng0https://orcid.org/0009-0003-0880-4927Mingyong Xin1Junwei Zhang2Fu Zou3Qingqing Zhao4Electric Power Research Institute of Guizhou Power Grid Co. Ltd., Guiyang, ChinaElectric Power Research Institute of Guizhou Power Grid Co. Ltd., Guiyang, ChinaElectric Power Research Institute of Guizhou Power Grid Co. Ltd., Guiyang, ChinaElectric Power Research Institute of Guizhou Power Grid Co. Ltd., Guiyang, ChinaElectric Power Research Institute of Guizhou Power Grid Co. Ltd., Guiyang, ChinaBad weather, such as rain, snow, and fog, will reduce the quality of the image acquired, and it will affect the performance of many related visual fields. The existing researches on image restoration under severe weather either focus on the restoration under a certain kind of weather, which cannot be generalized under different weather, or need to introduce additional model structures, which increases the burden of practical applications. Therefore, we explores an integrated image restoration framework, which can restore images under different adverse weather effects. Specifically, the model is trained in two stages. In the pre-training stage, a basic general network that can deal with multiple weather is trained by supervised learning. In the fine-tuning stage, soft prompts are introduced to stimulate the model’s ability to cope with different weather, so as to enhance the generalization ability. Specifically, the hidden information of the prompts is explored by low-rank decomposition, and the contras loss is added to prompt the prompts to converge on similar tasks. Furthermore, we address the distribution shift problem by aligning out-of-distribution (OOD) test sample statistics with those of the source data using test-time prompt tuning. Finally, we use the evaluation metrics, PSNR and SSIM, to evaluate the proposed method under four tasks: rain removal, snow removal, fog removal, and raindrop removal. The results demonstrate that the proposed method achieves superior performance compared to the existing state-of-the-art.https://ieeexplore.ieee.org/document/10810379/Transformer architecturepretrain-finetuningsoft promptscontrast learningdistribution alignment strategy |
| spellingShingle | Lingwen Meng Mingyong Xin Junwei Zhang Fu Zou Qingqing Zhao Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors IEEE Access Transformer architecture pretrain-finetuning soft prompts contrast learning distribution alignment strategy |
| title | Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors |
| title_full | Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors |
| title_fullStr | Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors |
| title_full_unstemmed | Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors |
| title_short | Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors |
| title_sort | multi task image restoration algorithm under different weather influence factors |
| topic | Transformer architecture pretrain-finetuning soft prompts contrast learning distribution alignment strategy |
| url | https://ieeexplore.ieee.org/document/10810379/ |
| work_keys_str_mv | AT lingwenmeng multitaskimagerestorationalgorithmunderdifferentweatherinfluencefactors AT mingyongxin multitaskimagerestorationalgorithmunderdifferentweatherinfluencefactors AT junweizhang multitaskimagerestorationalgorithmunderdifferentweatherinfluencefactors AT fuzou multitaskimagerestorationalgorithmunderdifferentweatherinfluencefactors AT qingqingzhao multitaskimagerestorationalgorithmunderdifferentweatherinfluencefactors |