MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks
In applications such as satellite remote sensing and aerial photography, imaging equipment must capture brightness information of different ground scenes within a restricted dynamic range. Due to camera sensor limitations, captured images can represent only a portion of such information, which resul...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-09-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/18/3501 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850259698546114560 |
|---|---|
| author | Sibo Yu Kun Wu Guang Zhang Wanhong Yan Xiaodong Wang Chen Tao |
| author_facet | Sibo Yu Kun Wu Guang Zhang Wanhong Yan Xiaodong Wang Chen Tao |
| author_sort | Sibo Yu |
| collection | DOAJ |
| description | In applications such as satellite remote sensing and aerial photography, imaging equipment must capture brightness information of different ground scenes within a restricted dynamic range. Due to camera sensor limitations, captured images can represent only a portion of such information, which results in lower resolution and lower dynamic range compared with real scenes. Image super resolution (SR) and multiple-exposure image fusion (MEF) are commonly employed technologies to address these issues. Nonetheless, these two problems are often researched in separate directions. In this paper, we propose MEFSR-GAN: an end-to-end framework based on generative adversarial networks that simultaneously combines super-resolution and multiple-exposure fusion. MEFSR-GAN includes a generator and two discriminators. The generator network consists of two parallel sub-networks for under-exposure and over-exposure, each containing a feature extraction block (FEB), a super-resolution block (SRB), and several multiple-exposure feedback blocks (MEFBs). It processes low-resolution under- and over-exposed images to produce high-resolution high dynamic range (HDR) images. These images are evaluated by two discriminator networks, driving the generator to generate realistic high-resolution HDR outputs through multi-goal training. Extensive qualitative and quantitative experiments were conducted on the SICE dataset, yielding a PSNR of 24.821 and an SSIM of 0.896 for 2× upscaling. These results demonstrate that MEFSR-GAN outperforms existing methods in terms of both visual effects and objective evaluation metrics, thereby establishing itself as a state-of-the-art technology. |
| format | Article |
| id | doaj-art-41ed2bca8e1c47f59d8c1aa059dd78ca |
| institution | OA Journals |
| issn | 2072-4292 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj-art-41ed2bca8e1c47f59d8c1aa059dd78ca2025-08-20T01:55:49ZengMDPI AGRemote Sensing2072-42922024-09-011618350110.3390/rs16183501MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial NetworksSibo Yu0Kun Wu1Guang Zhang2Wanhong Yan3Xiaodong Wang4Chen Tao5Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaIn applications such as satellite remote sensing and aerial photography, imaging equipment must capture brightness information of different ground scenes within a restricted dynamic range. Due to camera sensor limitations, captured images can represent only a portion of such information, which results in lower resolution and lower dynamic range compared with real scenes. Image super resolution (SR) and multiple-exposure image fusion (MEF) are commonly employed technologies to address these issues. Nonetheless, these two problems are often researched in separate directions. In this paper, we propose MEFSR-GAN: an end-to-end framework based on generative adversarial networks that simultaneously combines super-resolution and multiple-exposure fusion. MEFSR-GAN includes a generator and two discriminators. The generator network consists of two parallel sub-networks for under-exposure and over-exposure, each containing a feature extraction block (FEB), a super-resolution block (SRB), and several multiple-exposure feedback blocks (MEFBs). It processes low-resolution under- and over-exposed images to produce high-resolution high dynamic range (HDR) images. These images are evaluated by two discriminator networks, driving the generator to generate realistic high-resolution HDR outputs through multi-goal training. Extensive qualitative and quantitative experiments were conducted on the SICE dataset, yielding a PSNR of 24.821 and an SSIM of 0.896 for 2× upscaling. These results demonstrate that MEFSR-GAN outperforms existing methods in terms of both visual effects and objective evaluation metrics, thereby establishing itself as a state-of-the-art technology.https://www.mdpi.com/2072-4292/16/18/3501multitask networkssuper resolutionmultiple exposure fusiongenerative adversarial networks |
| spellingShingle | Sibo Yu Kun Wu Guang Zhang Wanhong Yan Xiaodong Wang Chen Tao MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks Remote Sensing multitask networks super resolution multiple exposure fusion generative adversarial networks |
| title | MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks |
| title_full | MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks |
| title_fullStr | MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks |
| title_full_unstemmed | MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks |
| title_short | MEFSR-GAN: A Multi-Exposure Feedback and Super-Resolution Multitask Network via Generative Adversarial Networks |
| title_sort | mefsr gan a multi exposure feedback and super resolution multitask network via generative adversarial networks |
| topic | multitask networks super resolution multiple exposure fusion generative adversarial networks |
| url | https://www.mdpi.com/2072-4292/16/18/3501 |
| work_keys_str_mv | AT siboyu mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks AT kunwu mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks AT guangzhang mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks AT wanhongyan mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks AT xiaodongwang mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks AT chentao mefsrganamultiexposurefeedbackandsuperresolutionmultitasknetworkviagenerativeadversarialnetworks |