HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis
Face sketch-to-photo synthesis is crucial in law enforcement, converting forensic sketches into RGB images for criminal database matching. A major challenge is ensuring accurate color representation in synthesized images to avoid identification error caused by mismatched skin or eye color. However,...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10916652/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850209860783702016 |
|---|---|
| author | Yeji Choi Haksub Kim Kwanghoon Sohn Ig-Jae Kim |
| author_facet | Yeji Choi Haksub Kim Kwanghoon Sohn Ig-Jae Kim |
| author_sort | Yeji Choi |
| collection | DOAJ |
| description | Face sketch-to-photo synthesis is crucial in law enforcement, converting forensic sketches into RGB images for criminal database matching. A major challenge is ensuring accurate color representation in synthesized images to avoid identification error caused by mismatched skin or eye color. However, direct sketch-to-photo translation struggles with proper color representation as it relies solely on grayscale sketches. While recent text-guided generative methods show promise for style adjustment based on text descriptions, they often produce mixed or exaggerated colors due to using a single representation for text prompts containing multiple entangled attributes. To address these challenges, we propose Hierarchical Text-guided Stylization (HiTS), a novel identity-preserving face sketch-to-photo synthesis method. HiTS categorizes text descriptions into intrinsic and mutable attributes, capturing both global and local color features. Using an encoder-decoder architecture, the encoder extracts global features from intrinsic attributes, while the decoder refines local styles via a semantic-textual embedding map. This map integrates text embeddings with facial parsing masks, enabling precise style adjustments for each facial component, even in small regions. Both quantitative and qualitative results demonstrate that HiTS achieves fine-grained stylization while preserving identity, leading to improved face recognition accuracy. |
| format | Article |
| id | doaj-art-0fe3a08bce0747a8bf18035b742b877e |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-0fe3a08bce0747a8bf18035b742b877e2025-08-20T02:09:55ZengIEEEIEEE Access2169-35362025-01-0113508855089410.1109/ACCESS.2025.354910210916652HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo SynthesisYeji Choi0https://orcid.org/0009-0001-8797-1576Haksub Kim1https://orcid.org/0000-0002-8780-9747Kwanghoon Sohn2https://orcid.org/0000-0002-3715-0331Ig-Jae Kim3https://orcid.org/0000-0002-2741-7047School of Electrical and Electronic Engineering, Yonsei University, Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology, Seoul, South KoreaSchool of Electrical and Electronic Engineering, Yonsei University, Seoul, South KoreaArtificial Intelligence and Robotics Institute, Korea Institute of Science and Technology, Seoul, South KoreaFace sketch-to-photo synthesis is crucial in law enforcement, converting forensic sketches into RGB images for criminal database matching. A major challenge is ensuring accurate color representation in synthesized images to avoid identification error caused by mismatched skin or eye color. However, direct sketch-to-photo translation struggles with proper color representation as it relies solely on grayscale sketches. While recent text-guided generative methods show promise for style adjustment based on text descriptions, they often produce mixed or exaggerated colors due to using a single representation for text prompts containing multiple entangled attributes. To address these challenges, we propose Hierarchical Text-guided Stylization (HiTS), a novel identity-preserving face sketch-to-photo synthesis method. HiTS categorizes text descriptions into intrinsic and mutable attributes, capturing both global and local color features. Using an encoder-decoder architecture, the encoder extracts global features from intrinsic attributes, while the decoder refines local styles via a semantic-textual embedding map. This map integrates text embeddings with facial parsing masks, enabling precise style adjustments for each facial component, even in small regions. Both quantitative and qualitative results demonstrate that HiTS achieves fine-grained stylization while preserving identity, leading to improved face recognition accuracy.https://ieeexplore.ieee.org/document/10916652/Face sketch-to-photo synthesisgenerative adversarial network (GAN)text-guided image generation |
| spellingShingle | Yeji Choi Haksub Kim Kwanghoon Sohn Ig-Jae Kim HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis IEEE Access Face sketch-to-photo synthesis generative adversarial network (GAN) text-guided image generation |
| title | HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis |
| title_full | HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis |
| title_fullStr | HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis |
| title_full_unstemmed | HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis |
| title_short | HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis |
| title_sort | hits hierarchical text guided stylization for face sketch to photo synthesis |
| topic | Face sketch-to-photo synthesis generative adversarial network (GAN) text-guided image generation |
| url | https://ieeexplore.ieee.org/document/10916652/ |
| work_keys_str_mv | AT yejichoi hitshierarchicaltextguidedstylizationforfacesketchtophotosynthesis AT haksubkim hitshierarchicaltextguidedstylizationforfacesketchtophotosynthesis AT kwanghoonsohn hitshierarchicaltextguidedstylizationforfacesketchtophotosynthesis AT igjaekim hitshierarchicaltextguidedstylizationforfacesketchtophotosynthesis |