Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images
In spectral imaging, the constraints imposed by hardware often lead to a limited spatial resolution within spectral filter array images. On the other hand, the process of demosaicking is challenging due to intricate filter patterns and a strong spectral cross correlation. Moreover, demosaicking and...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10838565/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832583240793194496 |
|---|---|
| author | Abdelhamid N. Fsian Jean-Baptiste Thomas Jon Y. Hardeberg Pierre Gouton |
| author_facet | Abdelhamid N. Fsian Jean-Baptiste Thomas Jon Y. Hardeberg Pierre Gouton |
| author_sort | Abdelhamid N. Fsian |
| collection | DOAJ |
| description | In spectral imaging, the constraints imposed by hardware often lead to a limited spatial resolution within spectral filter array images. On the other hand, the process of demosaicking is challenging due to intricate filter patterns and a strong spectral cross correlation. Moreover, demosaicking and super resolution are usually approached independently, overlooking the potential advantages of a joint solution. To this end, we use a two-branch framework, namely a pseudo-panchromatic image network and a pre-demosaicking sub-branch coupled with a novel deep residual demosaicking and super resolution module. This holistic approach ensures a more coherent and optimized restoration process, mitigating the risk of error accumulation and preserving image quality throughout the reconstruction pipeline. Our experimental results underscore the efficacy of the proposed network, showcasing an improvement of performance both qualitatively and quantitatively when compared to the sequential combination of state-of-the-art demosaicking and super resolution. With our proposed method, we obtained on the ARAD-1K dataset an average PSNR of 48.02 (dB) for domosaicking only, equivalent to the best method of the state-of-the-art. Moreover, for joint demosaicking and super resolution our model averages 35.26 (dB) and 26.29 (dB), respectively for <inline-formula> <tex-math notation="LaTeX">$\times 2$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\times 4$ </tex-math></inline-formula> upscale, outperforming state-of-the-art sequential approach.The codes and datasets are available at <uri>https://github.com/HamidFsian/DRDmSR</uri>. |
| format | Article |
| id | doaj-art-8e5418745ecb4b6c99906c1b2e130166 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-8e5418745ecb4b6c99906c1b2e1301662025-01-29T00:01:19ZengIEEEIEEE Access2169-35362025-01-0113162081622210.1109/ACCESS.2025.352875310838565Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array ImagesAbdelhamid N. Fsian0https://orcid.org/0009-0006-5388-7428Jean-Baptiste Thomas1Jon Y. Hardeberg2https://orcid.org/0000-0003-1150-2498Pierre Gouton3Department Informatique, Electronique, Mécanique (IEM), Imagerie et Vision Artificielle (ImVIA) Laboratory, Université de Bourgogne, Dijon, FranceDepartment Informatique, Electronique, Mécanique (IEM), Imagerie et Vision Artificielle (ImVIA) Laboratory, Université de Bourgogne, Dijon, FranceColourlab, Department of Computer Science, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwayDepartment Informatique, Electronique, Mécanique (IEM), Imagerie et Vision Artificielle (ImVIA) Laboratory, Université de Bourgogne, Dijon, FranceIn spectral imaging, the constraints imposed by hardware often lead to a limited spatial resolution within spectral filter array images. On the other hand, the process of demosaicking is challenging due to intricate filter patterns and a strong spectral cross correlation. Moreover, demosaicking and super resolution are usually approached independently, overlooking the potential advantages of a joint solution. To this end, we use a two-branch framework, namely a pseudo-panchromatic image network and a pre-demosaicking sub-branch coupled with a novel deep residual demosaicking and super resolution module. This holistic approach ensures a more coherent and optimized restoration process, mitigating the risk of error accumulation and preserving image quality throughout the reconstruction pipeline. Our experimental results underscore the efficacy of the proposed network, showcasing an improvement of performance both qualitatively and quantitatively when compared to the sequential combination of state-of-the-art demosaicking and super resolution. With our proposed method, we obtained on the ARAD-1K dataset an average PSNR of 48.02 (dB) for domosaicking only, equivalent to the best method of the state-of-the-art. Moreover, for joint demosaicking and super resolution our model averages 35.26 (dB) and 26.29 (dB), respectively for <inline-formula> <tex-math notation="LaTeX">$\times 2$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\times 4$ </tex-math></inline-formula> upscale, outperforming state-of-the-art sequential approach.The codes and datasets are available at <uri>https://github.com/HamidFsian/DRDmSR</uri>.https://ieeexplore.ieee.org/document/10838565/Spectral imagingdemosaickingsuper resolutiondeep learningpseudo-panchromatic imagespectral filter array |
| spellingShingle | Abdelhamid N. Fsian Jean-Baptiste Thomas Jon Y. Hardeberg Pierre Gouton Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images IEEE Access Spectral imaging demosaicking super resolution deep learning pseudo-panchromatic image spectral filter array |
| title | Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images |
| title_full | Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images |
| title_fullStr | Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images |
| title_full_unstemmed | Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images |
| title_short | Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images |
| title_sort | deep joint demosaicking and super resolution for spectral filter array images |
| topic | Spectral imaging demosaicking super resolution deep learning pseudo-panchromatic image spectral filter array |
| url | https://ieeexplore.ieee.org/document/10838565/ |
| work_keys_str_mv | AT abdelhamidnfsian deepjointdemosaickingandsuperresolutionforspectralfilterarrayimages AT jeanbaptistethomas deepjointdemosaickingandsuperresolutionforspectralfilterarrayimages AT jonyhardeberg deepjointdemosaickingandsuperresolutionforspectralfilterarrayimages AT pierregouton deepjointdemosaickingandsuperresolutionforspectralfilterarrayimages |