Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network
Light Field Angular Super-Resolution (LFASR) addresses the issue where Light Field (LF) images can not simultaneously achieve both high spatial and angular resolution due to the limited resolution of optical sensors. Since Spatial-Angular Correlation (SAC) features are closely related to the structu...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/4/991 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849722261843476480 |
|---|---|
| author | Daichuan Li Rui Zhong Yungang Yang |
| author_facet | Daichuan Li Rui Zhong Yungang Yang |
| author_sort | Daichuan Li |
| collection | DOAJ |
| description | Light Field Angular Super-Resolution (LFASR) addresses the issue where Light Field (LF) images can not simultaneously achieve both high spatial and angular resolution due to the limited resolution of optical sensors. Since Spatial-Angular Correlation (SAC) features are closely related to the structure of LF images, its accurate and complete extraction is crucial for the quality of LF images reconstructed by the LFASR method based on Deep Neural Networks (DNNs). In low-angular resolution LF images, SAC features must be extracted from a limited number of pixels that are at a great distance from each other and exhibit strong correlations. However, existing LFASR methods based on DNNs fail to extract SAC features accurately and completely. Due to the limited receptive field, methods based on regular Convolutional Neural Networks (CNNs) are unable to capture SAC features from distant pixels, leading to incomplete SAC feature extraction. On the other hand, methods based on large convolution kernels and attention mechanisms use an excessive number of pixels to extract SAC features, resulting in insufficient accuracy in extracted SAC features. To solve this problem, we introduce Deformable Convolutional Network (DCN), which adaptively changes the position of limited sampling point using offsets, so as to extract SAC from distant pixels. In addition, in order to make the offset of DCN more accurate and further improve the accuracy of SAC features, we also propose a Multi-Maximum-Offsets Fusion DCN (MMOF-DCN). MMOF-DCN can reduce the exploration range of finding the desired offset, thereby improving the offset finding efficiency. Experiment results show that our proposed method has advantages in real-world dataset and synthetic dataset. The PSNR value in synthetic dataset which have large disparity is improved by 0.45 dB compared to existing methods. |
| format | Article |
| id | doaj-art-e1e9e859597b43529e87a1ad6838cf78 |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-e1e9e859597b43529e87a1ad6838cf782025-08-20T03:11:22ZengMDPI AGSensors1424-82202025-02-0125499110.3390/s25040991Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional NetworkDaichuan Li0Rui Zhong1Yungang Yang2School of Computer Science, Central China Normal University, Wuhan 430079, ChinaSchool of Computer Science, Central China Normal University, Wuhan 430079, ChinaSchool of Computer Science, Central China Normal University, Wuhan 430079, ChinaLight Field Angular Super-Resolution (LFASR) addresses the issue where Light Field (LF) images can not simultaneously achieve both high spatial and angular resolution due to the limited resolution of optical sensors. Since Spatial-Angular Correlation (SAC) features are closely related to the structure of LF images, its accurate and complete extraction is crucial for the quality of LF images reconstructed by the LFASR method based on Deep Neural Networks (DNNs). In low-angular resolution LF images, SAC features must be extracted from a limited number of pixels that are at a great distance from each other and exhibit strong correlations. However, existing LFASR methods based on DNNs fail to extract SAC features accurately and completely. Due to the limited receptive field, methods based on regular Convolutional Neural Networks (CNNs) are unable to capture SAC features from distant pixels, leading to incomplete SAC feature extraction. On the other hand, methods based on large convolution kernels and attention mechanisms use an excessive number of pixels to extract SAC features, resulting in insufficient accuracy in extracted SAC features. To solve this problem, we introduce Deformable Convolutional Network (DCN), which adaptively changes the position of limited sampling point using offsets, so as to extract SAC from distant pixels. In addition, in order to make the offset of DCN more accurate and further improve the accuracy of SAC features, we also propose a Multi-Maximum-Offsets Fusion DCN (MMOF-DCN). MMOF-DCN can reduce the exploration range of finding the desired offset, thereby improving the offset finding efficiency. Experiment results show that our proposed method has advantages in real-world dataset and synthetic dataset. The PSNR value in synthetic dataset which have large disparity is improved by 0.45 dB compared to existing methods.https://www.mdpi.com/1424-8220/25/4/991light fieldangular super-resolutionoptical sensorsreconstructdeep neural network |
| spellingShingle | Daichuan Li Rui Zhong Yungang Yang Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network Sensors light field angular super-resolution optical sensors reconstruct deep neural network |
| title | Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network |
| title_full | Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network |
| title_fullStr | Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network |
| title_full_unstemmed | Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network |
| title_short | Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network |
| title_sort | light field angular super resolution via spatial angular correlation extracted by deformable convolutional network |
| topic | light field angular super-resolution optical sensors reconstruct deep neural network |
| url | https://www.mdpi.com/1424-8220/25/4/991 |
| work_keys_str_mv | AT daichuanli lightfieldangularsuperresolutionviaspatialangularcorrelationextractedbydeformableconvolutionalnetwork AT ruizhong lightfieldangularsuperresolutionviaspatialangularcorrelationextractedbydeformableconvolutionalnetwork AT yungangyang lightfieldangularsuperresolutionviaspatialangularcorrelationextractedbydeformableconvolutionalnetwork |