Architectural facade design with style and structural features using stable diffusion model
With advancements in digital technology, the field of architectural design has increasingly embraced data and algorithms to enhance design efficiency and quality. Recent advancements in text-to-image (T2I) generation models have enabled the creation of images that correspond to textual descriptions....
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-12-01
|
| Series: | Journal of Intelligent Construction |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/JIC.2024.9180034 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846171310232698880 |
|---|---|
| author | Minghao Wen Dong Liang Haibo Ye Huawei Tu |
| author_facet | Minghao Wen Dong Liang Haibo Ye Huawei Tu |
| author_sort | Minghao Wen |
| collection | DOAJ |
| description | With advancements in digital technology, the field of architectural design has increasingly embraced data and algorithms to enhance design efficiency and quality. Recent advancements in text-to-image (T2I) generation models have enabled the creation of images that correspond to textual descriptions. However, textual descriptions struggle to capture essential style characteristics in style images. In this study, we proposed a method for architectural facade design based on the stable diffusion model (SDM) that combined stylistic images or keywords as input with the structural conditions of content images to generate images with both stylistic and architectural features. By employing the constrastive language-image pre-training (CLIP) image encoder to convert the style image into its initial image embedding and feature extraction from multilayer cross-attention and training optimization to obtain a pretrained image embedding, the proposed method extracts stylistic features from style images and converts them into corresponding embeddings. This process enables the generated images to embody stylistic features and artistic semantic information. Furthermore, the T2I adapter model is employed to use the architectural structure of content images as conditional guidance, thereby ensuring that the generated images exhibit the corresponding structural features. By leveraging these two aspects, the proposed method can decorate architecture with stylistic features from stylistic images while preserving the architectural structure features of content images, resulting in images that reflect the content images after style transformation. Our method is mainly used in architectural design applications. It was capable of generating facade images from flat design drawings, three-dimensional (3D) architectural models, and hand-drawn sketches and has achieved commendable results. |
| format | Article |
| id | doaj-art-0d1c443eb0724c6ea01cbe4b3c9f6cfa |
| institution | Kabale University |
| issn | 2958-3861 2958-2652 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | Journal of Intelligent Construction |
| spelling | doaj-art-0d1c443eb0724c6ea01cbe4b3c9f6cfa2024-11-11T03:37:19ZengTsinghua University PressJournal of Intelligent Construction2958-38612958-26522024-12-0124918003410.26599/JIC.2024.9180034Architectural facade design with style and structural features using stable diffusion modelMinghao Wen0Dong Liang1Haibo Ye2Huawei Tu3College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaDepartment of Computer Science and Information Technology, La Trobe University, Melbourne 3086, AustraliaWith advancements in digital technology, the field of architectural design has increasingly embraced data and algorithms to enhance design efficiency and quality. Recent advancements in text-to-image (T2I) generation models have enabled the creation of images that correspond to textual descriptions. However, textual descriptions struggle to capture essential style characteristics in style images. In this study, we proposed a method for architectural facade design based on the stable diffusion model (SDM) that combined stylistic images or keywords as input with the structural conditions of content images to generate images with both stylistic and architectural features. By employing the constrastive language-image pre-training (CLIP) image encoder to convert the style image into its initial image embedding and feature extraction from multilayer cross-attention and training optimization to obtain a pretrained image embedding, the proposed method extracts stylistic features from style images and converts them into corresponding embeddings. This process enables the generated images to embody stylistic features and artistic semantic information. Furthermore, the T2I adapter model is employed to use the architectural structure of content images as conditional guidance, thereby ensuring that the generated images exhibit the corresponding structural features. By leveraging these two aspects, the proposed method can decorate architecture with stylistic features from stylistic images while preserving the architectural structure features of content images, resulting in images that reflect the content images after style transformation. Our method is mainly used in architectural design applications. It was capable of generating facade images from flat design drawings, three-dimensional (3D) architectural models, and hand-drawn sketches and has achieved commendable results.https://www.sciopen.com/article/10.26599/JIC.2024.9180034architectural designdigital facade generationstyle transferstable diffusiont2i-adapter |
| spellingShingle | Minghao Wen Dong Liang Haibo Ye Huawei Tu Architectural facade design with style and structural features using stable diffusion model Journal of Intelligent Construction architectural design digital facade generation style transfer stable diffusion t2i-adapter |
| title | Architectural facade design with style and structural features using stable diffusion model |
| title_full | Architectural facade design with style and structural features using stable diffusion model |
| title_fullStr | Architectural facade design with style and structural features using stable diffusion model |
| title_full_unstemmed | Architectural facade design with style and structural features using stable diffusion model |
| title_short | Architectural facade design with style and structural features using stable diffusion model |
| title_sort | architectural facade design with style and structural features using stable diffusion model |
| topic | architectural design digital facade generation style transfer stable diffusion t2i-adapter |
| url | https://www.sciopen.com/article/10.26599/JIC.2024.9180034 |
| work_keys_str_mv | AT minghaowen architecturalfacadedesignwithstyleandstructuralfeaturesusingstablediffusionmodel AT dongliang architecturalfacadedesignwithstyleandstructuralfeaturesusingstablediffusionmodel AT haiboye architecturalfacadedesignwithstyleandstructuralfeaturesusingstablediffusionmodel AT huaweitu architecturalfacadedesignwithstyleandstructuralfeaturesusingstablediffusionmodel |