Foveated Denoising for Ray Tracing Rendering
Foveated graphics allocate computational load in light of the non-uniform nature of the human visual system (HVS), accelerating the scene rendering of virtual reality (VR). Although the asymmetrical spatial sensitivity of the HVS, such as visual acuity, is widely recognized and exploited to improve...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10804172/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841533400059478016 |
|---|---|
| author | Daiyun Guo Yan Zhang Xubo Yang |
| author_facet | Daiyun Guo Yan Zhang Xubo Yang |
| author_sort | Daiyun Guo |
| collection | DOAJ |
| description | Foveated graphics allocate computational load in light of the non-uniform nature of the human visual system (HVS), accelerating the scene rendering of virtual reality (VR). Although the asymmetrical spatial sensitivity of the HVS, such as visual acuity, is widely recognized and exploited to improve the rendering performance of VR, the variation of temporal contrast sensitivity in FOVs is less explored. In this paper, we quantify the non-uniform response of the HVS to flickering noise in the field of views (FOVs) by an SSIM-based model, where the sensitivity of each visual field is proportional to the tolerated variation of local SSIM per frame. Our pilot experiment reveals that denoising requirements vary significantly across the FOV, with central vision demanding a more stable SSIM variation. Based on these findings, a foveated denoising method is proposed. The central vision within 18.5° is rendered with deep learning (DL) based denoising, and the periphery is rendered with temporal anti-aliasing (TAA). A user study is conducted with VR scenes rendered by using ray tracing. The experiment results demonstrate the foveated denoising method provides perceptually comparable image quality to the global DL-based denoising method while saving at least 40.22% computational cost in VR. |
| format | Article |
| id | doaj-art-41631163bffb4591aac424bed076bfd7 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-41631163bffb4591aac424bed076bfd72025-01-16T00:01:56ZengIEEEIEEE Access2169-35362024-01-011219801519802710.1109/ACCESS.2024.351716310804172Foveated Denoising for Ray Tracing RenderingDaiyun Guo0https://orcid.org/0009-0008-4575-6262Yan Zhang1https://orcid.org/0000-0002-7549-4725Xubo Yang2https://orcid.org/0000-0001-5378-4003School of Software, Shanghai Jiao Tong University, Shanghai, ChinaSchool of Software, Shanghai Jiao Tong University, Shanghai, ChinaSchool of Software, Shanghai Jiao Tong University, Shanghai, ChinaFoveated graphics allocate computational load in light of the non-uniform nature of the human visual system (HVS), accelerating the scene rendering of virtual reality (VR). Although the asymmetrical spatial sensitivity of the HVS, such as visual acuity, is widely recognized and exploited to improve the rendering performance of VR, the variation of temporal contrast sensitivity in FOVs is less explored. In this paper, we quantify the non-uniform response of the HVS to flickering noise in the field of views (FOVs) by an SSIM-based model, where the sensitivity of each visual field is proportional to the tolerated variation of local SSIM per frame. Our pilot experiment reveals that denoising requirements vary significantly across the FOV, with central vision demanding a more stable SSIM variation. Based on these findings, a foveated denoising method is proposed. The central vision within 18.5° is rendered with deep learning (DL) based denoising, and the periphery is rendered with temporal anti-aliasing (TAA). A user study is conducted with VR scenes rendered by using ray tracing. The experiment results demonstrate the foveated denoising method provides perceptually comparable image quality to the global DL-based denoising method while saving at least 40.22% computational cost in VR.https://ieeexplore.ieee.org/document/10804172/Virtual realityfoveated renderingray tracingdenoising |
| spellingShingle | Daiyun Guo Yan Zhang Xubo Yang Foveated Denoising for Ray Tracing Rendering IEEE Access Virtual reality foveated rendering ray tracing denoising |
| title | Foveated Denoising for Ray Tracing Rendering |
| title_full | Foveated Denoising for Ray Tracing Rendering |
| title_fullStr | Foveated Denoising for Ray Tracing Rendering |
| title_full_unstemmed | Foveated Denoising for Ray Tracing Rendering |
| title_short | Foveated Denoising for Ray Tracing Rendering |
| title_sort | foveated denoising for ray tracing rendering |
| topic | Virtual reality foveated rendering ray tracing denoising |
| url | https://ieeexplore.ieee.org/document/10804172/ |
| work_keys_str_mv | AT daiyunguo foveateddenoisingforraytracingrendering AT yanzhang foveateddenoisingforraytracingrendering AT xuboyang foveateddenoisingforraytracingrendering |