Real-time facial reconstruction and expression replacement based on neural radiation field
It is now possible to do high-fidelity 3D facial reconstruction and unique view synthesis thanks to the recent discovery of Neural Radiance Fields (NeRF), which has established its substantial importance in the field of 3D vision. However, the operational approaches that are now in use require a sig...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-12-01
|
| Series: | Systems and Soft Computing |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2772941925000031 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832586213402345472 |
|---|---|
| author | Shenning Zhang Hui Li Xuefeng Tian |
| author_facet | Shenning Zhang Hui Li Xuefeng Tian |
| author_sort | Shenning Zhang |
| collection | DOAJ |
| description | It is now possible to do high-fidelity 3D facial reconstruction and unique view synthesis thanks to the recent discovery of Neural Radiance Fields (NeRF), which has established its substantial importance in the field of 3D vision. However, the operational approaches that are now in use require a significant amount of human engagement, such as the need for users to provide semantic masks and the inconvenience of manual attribute searching for non-expert users. Our approach focuses on enabling the manipulation of NeRF-reconstructed faces with just a single text input. A scene manipulator, specifically a conditional version NeRF with deformable latent codes, is the first thing that this paper trains to accomplish this objective, in dynamic scenes, allowing facial deformations to be controlled through latent codes. However, to synthesize local deformations in a variety of contexts, it is not desirable to describe scene deformations using only a single latent coding. Therefore, this paper proposes a text-driven operation pipeline for facial reconstruction with NeRF, the development of an operating network that is capable of learning to represent scene changes using latent codes that vary at different spatial locations, and the integration of a WeChat mini-program to facilitate practical applications. This application approach enables even non-expert users to easily synthesize novel views. Our method has achieved a certain breakthrough in the field of 3D facial reconstruction, providing users with a simple and convenient text-driven operation approach. |
| format | Article |
| id | doaj-art-967df51759974abaa99c15359b248d90 |
| institution | Kabale University |
| issn | 2772-9419 |
| language | English |
| publishDate | 2025-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Systems and Soft Computing |
| spelling | doaj-art-967df51759974abaa99c15359b248d902025-01-26T05:05:24ZengElsevierSystems and Soft Computing2772-94192025-12-017200185Real-time facial reconstruction and expression replacement based on neural radiation fieldShenning Zhang0Hui Li1Xuefeng Tian2School of Advanced Manufacturing, Guangdong Songshan Polytechnic, Shaoguan 512000, ChinaSchool of Foreign Languages and International Business, Guangdong Songshan Polytechnic, Shaoguan Guangdong 512000, ChinaSchool of Advanced Manufacturing, Guangdong Songshan Polytechnic, Shaoguan 512000, China; Corresponding author.It is now possible to do high-fidelity 3D facial reconstruction and unique view synthesis thanks to the recent discovery of Neural Radiance Fields (NeRF), which has established its substantial importance in the field of 3D vision. However, the operational approaches that are now in use require a significant amount of human engagement, such as the need for users to provide semantic masks and the inconvenience of manual attribute searching for non-expert users. Our approach focuses on enabling the manipulation of NeRF-reconstructed faces with just a single text input. A scene manipulator, specifically a conditional version NeRF with deformable latent codes, is the first thing that this paper trains to accomplish this objective, in dynamic scenes, allowing facial deformations to be controlled through latent codes. However, to synthesize local deformations in a variety of contexts, it is not desirable to describe scene deformations using only a single latent coding. Therefore, this paper proposes a text-driven operation pipeline for facial reconstruction with NeRF, the development of an operating network that is capable of learning to represent scene changes using latent codes that vary at different spatial locations, and the integration of a WeChat mini-program to facilitate practical applications. This application approach enables even non-expert users to easily synthesize novel views. Our method has achieved a certain breakthrough in the field of 3D facial reconstruction, providing users with a simple and convenient text-driven operation approach.http://www.sciencedirect.com/science/article/pii/S2772941925000031Neural radiation fieldFacial reconstructionexpression replacement3D facial reconstruction |
| spellingShingle | Shenning Zhang Hui Li Xuefeng Tian Real-time facial reconstruction and expression replacement based on neural radiation field Systems and Soft Computing Neural radiation field Facial reconstruction expression replacement 3D facial reconstruction |
| title | Real-time facial reconstruction and expression replacement based on neural radiation field |
| title_full | Real-time facial reconstruction and expression replacement based on neural radiation field |
| title_fullStr | Real-time facial reconstruction and expression replacement based on neural radiation field |
| title_full_unstemmed | Real-time facial reconstruction and expression replacement based on neural radiation field |
| title_short | Real-time facial reconstruction and expression replacement based on neural radiation field |
| title_sort | real time facial reconstruction and expression replacement based on neural radiation field |
| topic | Neural radiation field Facial reconstruction expression replacement 3D facial reconstruction |
| url | http://www.sciencedirect.com/science/article/pii/S2772941925000031 |
| work_keys_str_mv | AT shenningzhang realtimefacialreconstructionandexpressionreplacementbasedonneuralradiationfield AT huili realtimefacialreconstructionandexpressionreplacementbasedonneuralradiationfield AT xuefengtian realtimefacialreconstructionandexpressionreplacementbasedonneuralradiationfield |