Diversified Image Inpainting With Transformers and Denoising Iterative Refinement
Image inpainting is a long-standing key problem in the field of computer vision, which aims to fill the missing parts of an image with visually realistic and semantically appropriate content. For a long time, in the research work at home and abroad, how to generate diverse and realistic images is a...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10788702/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850113945598164992 |
|---|---|
| author | Shuzhen Xu Wenlong Xiang Cuicui Lv Shuo Wang Guanhua Liu |
| author_facet | Shuzhen Xu Wenlong Xiang Cuicui Lv Shuo Wang Guanhua Liu |
| author_sort | Shuzhen Xu |
| collection | DOAJ |
| description | Image inpainting is a long-standing key problem in the field of computer vision, which aims to fill the missing parts of an image with visually realistic and semantically appropriate content. For a long time, in the research work at home and abroad, how to generate diverse and realistic images is a dilemma faced by image inpainting. With the continuous iteration of deep learning technology, transformer feature extraction model and a new generation paradigm, diffusion model, are emerging in vision tasks. Attention-based transformer models can effectively model long-distance dependencies and flexibly design output content. The diffusion model is stable in training, and the quality of its generated images is already better than that of generative adversarial networks. We decompose the inpainting problem into two key steps: diversified pre-generation and high-resolution reconstruction. Firstly, referring to the discretized pixel set, a transformer information association model is designed to sample from the granularity of pixel values, so as to obtain low-resolution results with diverse appearance. Then, a denoising diffusion model is used to reconstruct the high-resolution image, which is a conditional and iterative refinement process. Ultimately, we achieve a set of image restoration methods that produce diverse results and support high-fidelity output. |
| format | Article |
| id | doaj-art-53ceb8e2ef7645e19632db6740004f6c |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-53ceb8e2ef7645e19632db6740004f6c2025-08-20T02:37:02ZengIEEEIEEE Access2169-35362024-01-011218706818708010.1109/ACCESS.2024.351493010788702Diversified Image Inpainting With Transformers and Denoising Iterative RefinementShuzhen Xu0https://orcid.org/0000-0003-0512-8276Wenlong Xiang1Cuicui Lv2https://orcid.org/0000-0002-5673-0729Shuo Wang3https://orcid.org/0009-0001-0768-7920Guanhua Liu4School of Computer and Control Engineering, Yantai University, Yantai, ChinaSchool of Computer and Control Engineering, Yantai University, Yantai, ChinaSchool of Computer and Control Engineering, Yantai University, Yantai, ChinaSchool of Computer and Control Engineering, Yantai University, Yantai, ChinaCollege of Technology and Data, Yantai Nanshan University, Yantai, Longkou, ChinaImage inpainting is a long-standing key problem in the field of computer vision, which aims to fill the missing parts of an image with visually realistic and semantically appropriate content. For a long time, in the research work at home and abroad, how to generate diverse and realistic images is a dilemma faced by image inpainting. With the continuous iteration of deep learning technology, transformer feature extraction model and a new generation paradigm, diffusion model, are emerging in vision tasks. Attention-based transformer models can effectively model long-distance dependencies and flexibly design output content. The diffusion model is stable in training, and the quality of its generated images is already better than that of generative adversarial networks. We decompose the inpainting problem into two key steps: diversified pre-generation and high-resolution reconstruction. Firstly, referring to the discretized pixel set, a transformer information association model is designed to sample from the granularity of pixel values, so as to obtain low-resolution results with diverse appearance. Then, a denoising diffusion model is used to reconstruct the high-resolution image, which is a conditional and iterative refinement process. Ultimately, we achieve a set of image restoration methods that produce diverse results and support high-fidelity output.https://ieeexplore.ieee.org/document/10788702/Image inpaintingdiffusion modeltransformerdiversified inpainting results |
| spellingShingle | Shuzhen Xu Wenlong Xiang Cuicui Lv Shuo Wang Guanhua Liu Diversified Image Inpainting With Transformers and Denoising Iterative Refinement IEEE Access Image inpainting diffusion model transformer diversified inpainting results |
| title | Diversified Image Inpainting With Transformers and Denoising Iterative Refinement |
| title_full | Diversified Image Inpainting With Transformers and Denoising Iterative Refinement |
| title_fullStr | Diversified Image Inpainting With Transformers and Denoising Iterative Refinement |
| title_full_unstemmed | Diversified Image Inpainting With Transformers and Denoising Iterative Refinement |
| title_short | Diversified Image Inpainting With Transformers and Denoising Iterative Refinement |
| title_sort | diversified image inpainting with transformers and denoising iterative refinement |
| topic | Image inpainting diffusion model transformer diversified inpainting results |
| url | https://ieeexplore.ieee.org/document/10788702/ |
| work_keys_str_mv | AT shuzhenxu diversifiedimageinpaintingwithtransformersanddenoisingiterativerefinement AT wenlongxiang diversifiedimageinpaintingwithtransformersanddenoisingiterativerefinement AT cuicuilv diversifiedimageinpaintingwithtransformersanddenoisingiterativerefinement AT shuowang diversifiedimageinpaintingwithtransformersanddenoisingiterativerefinement AT guanhualiu diversifiedimageinpaintingwithtransformersanddenoisingiterativerefinement |