Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input
ABSTRACT Realistic human reconstruction embraces an extensive range of applications as depth sensors advance. However, current state‐of‐the‐art methods with RGB‐D input still suffer from artefacts, such as noisy surfaces, non‐human shapes, and depth ambiguity, especially for the invisible parts. The...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-06-01
|
| Series: | CAAI Transactions on Intelligence Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cit2.70009 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850121668872110080 |
|---|---|
| author | Pengpeng Liu Zhi Zeng Qisheng Wang Min Chen Guixuan Zhang |
| author_facet | Pengpeng Liu Zhi Zeng Qisheng Wang Min Chen Guixuan Zhang |
| author_sort | Pengpeng Liu |
| collection | DOAJ |
| description | ABSTRACT Realistic human reconstruction embraces an extensive range of applications as depth sensors advance. However, current state‐of‐the‐art methods with RGB‐D input still suffer from artefacts, such as noisy surfaces, non‐human shapes, and depth ambiguity, especially for the invisible parts. The authors observe the main issue is the lack of geometric semantics without using depth input priors fully. This paper focuses on improving the representation ability of implicit function, exploring an effective method to utilise depth‐related semantics effectively and efficiently. The proposed geometry‐enhanced implicit function enhances the geometric semantics with the extra voxel‐aligned features from point clouds, promoting the completion of missing parts for unseen regions while preserving the local details on the input. For incorporating multi‐scale pixel‐aligned and voxel‐aligned features, the authors use the Squeeze‐and‐Excitation attention to capture and fully use channel interdependencies. For the multi‐view reconstruction, the proposed depth‐enhanced attention explicitly excites the network to “sense” the geometric structure for a more reasonable feature aggregation. Experiments and results show that our method outperforms current RGB and depth‐based SOTA methods on the challenging data from Twindom and Thuman3.0, and achieves a detailed and completed human reconstruction, balancing performance and efficiency well. |
| format | Article |
| id | doaj-art-9b64e941c034478b93c47fb3e34cce90 |
| institution | OA Journals |
| issn | 2468-2322 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | CAAI Transactions on Intelligence Technology |
| spelling | doaj-art-9b64e941c034478b93c47fb3e34cce902025-08-20T02:35:01ZengWileyCAAI Transactions on Intelligence Technology2468-23222025-06-0110385887010.1049/cit2.70009Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D InputPengpeng Liu0Zhi Zeng1Qisheng Wang2Min Chen3Guixuan Zhang4Key Laboratory of Digital Rights Services Institute of Automation, Chinese Academy of Sciences Beijing ChinaBeijing University of Posts and Telecommunications Beijing ChinaHithink RoyalFlush Information Network Co. Ltd. Hangzhou ChinaHithink RoyalFlush Information Network Co. Ltd. Hangzhou ChinaBeijing University of Posts and Telecommunications Beijing ChinaABSTRACT Realistic human reconstruction embraces an extensive range of applications as depth sensors advance. However, current state‐of‐the‐art methods with RGB‐D input still suffer from artefacts, such as noisy surfaces, non‐human shapes, and depth ambiguity, especially for the invisible parts. The authors observe the main issue is the lack of geometric semantics without using depth input priors fully. This paper focuses on improving the representation ability of implicit function, exploring an effective method to utilise depth‐related semantics effectively and efficiently. The proposed geometry‐enhanced implicit function enhances the geometric semantics with the extra voxel‐aligned features from point clouds, promoting the completion of missing parts for unseen regions while preserving the local details on the input. For incorporating multi‐scale pixel‐aligned and voxel‐aligned features, the authors use the Squeeze‐and‐Excitation attention to capture and fully use channel interdependencies. For the multi‐view reconstruction, the proposed depth‐enhanced attention explicitly excites the network to “sense” the geometric structure for a more reasonable feature aggregation. Experiments and results show that our method outperforms current RGB and depth‐based SOTA methods on the challenging data from Twindom and Thuman3.0, and achieves a detailed and completed human reconstruction, balancing performance and efficiency well.https://doi.org/10.1049/cit2.70009deep implicit functiondepth‐enhanced attentiongeometry‐enhancedhuman reconstructionRGB‐D |
| spellingShingle | Pengpeng Liu Zhi Zeng Qisheng Wang Min Chen Guixuan Zhang Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input CAAI Transactions on Intelligence Technology deep implicit function depth‐enhanced attention geometry‐enhanced human reconstruction RGB‐D |
| title | Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input |
| title_full | Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input |
| title_fullStr | Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input |
| title_full_unstemmed | Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input |
| title_short | Geometry‐Enhanced Implicit Function for Detailed Clothed Human Reconstruction With RGB‐D Input |
| title_sort | geometry enhanced implicit function for detailed clothed human reconstruction with rgb d input |
| topic | deep implicit function depth‐enhanced attention geometry‐enhanced human reconstruction RGB‐D |
| url | https://doi.org/10.1049/cit2.70009 |
| work_keys_str_mv | AT pengpengliu geometryenhancedimplicitfunctionfordetailedclothedhumanreconstructionwithrgbdinput AT zhizeng geometryenhancedimplicitfunctionfordetailedclothedhumanreconstructionwithrgbdinput AT qishengwang geometryenhancedimplicitfunctionfordetailedclothedhumanreconstructionwithrgbdinput AT minchen geometryenhancedimplicitfunctionfordetailedclothedhumanreconstructionwithrgbdinput AT guixuanzhang geometryenhancedimplicitfunctionfordetailedclothedhumanreconstructionwithrgbdinput |