Multi‐stage image inpainting using improved partial convolutions
Abstract In recent years, deep learning models have dramatically influenced image inpainting. However, many existing studies still suffer from over‐smoothed or blurred textures when missing regions are large or contain rich visual details. To restore textures at a fine‐grained level, a multi‐stage i...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-10-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.13178 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850200371182436352 |
|---|---|
| author | Cheng Li Dan Xu Hao Zhang |
| author_facet | Cheng Li Dan Xu Hao Zhang |
| author_sort | Cheng Li |
| collection | DOAJ |
| description | Abstract In recent years, deep learning models have dramatically influenced image inpainting. However, many existing studies still suffer from over‐smoothed or blurred textures when missing regions are large or contain rich visual details. To restore textures at a fine‐grained level, a multi‐stage inpainting approach is proposed, which applies a series of partial inpainting modules as well as a progressive inpainting module to inpaint missing areas from their boundaries to the centre successively. Some improvements are made on the partial convolutions to reduce artifacts like blurriness, which require a convolution kernel to contain known pixels more than a certain proportion. Towards photorealistic inpainting results, the intermediate outputs from each stage are used to compute the loss. Finally, to facilitate the training process, a multi‐step training is designed that progressively adds inpainting modules to optimize the model. Experiments show that this method outperforms the current excellent techniques on the publicly available datasets: CelebA, Places2 and Paris StreetView. |
| format | Article |
| id | doaj-art-0da85b4d2d2c438f9576de894c9e42ec |
| institution | OA Journals |
| issn | 1751-9659 1751-9667 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj-art-0da85b4d2d2c438f9576de894c9e42ec2025-08-20T02:12:20ZengWileyIET Image Processing1751-96591751-96672024-10-0118123343335510.1049/ipr2.13178Multi‐stage image inpainting using improved partial convolutionsCheng Li0Dan Xu1Hao Zhang2School of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaSchool of Information Science and Engineering Yunnan University Kunming ChinaAbstract In recent years, deep learning models have dramatically influenced image inpainting. However, many existing studies still suffer from over‐smoothed or blurred textures when missing regions are large or contain rich visual details. To restore textures at a fine‐grained level, a multi‐stage inpainting approach is proposed, which applies a series of partial inpainting modules as well as a progressive inpainting module to inpaint missing areas from their boundaries to the centre successively. Some improvements are made on the partial convolutions to reduce artifacts like blurriness, which require a convolution kernel to contain known pixels more than a certain proportion. Towards photorealistic inpainting results, the intermediate outputs from each stage are used to compute the loss. Finally, to facilitate the training process, a multi‐step training is designed that progressively adds inpainting modules to optimize the model. Experiments show that this method outperforms the current excellent techniques on the publicly available datasets: CelebA, Places2 and Paris StreetView.https://doi.org/10.1049/ipr2.13178image inpaintingimage processingimage reconstructionmulti‐step training processpartial inpainting moduleprogressive inpainting module |
| spellingShingle | Cheng Li Dan Xu Hao Zhang Multi‐stage image inpainting using improved partial convolutions IET Image Processing image inpainting image processing image reconstruction multi‐step training process partial inpainting module progressive inpainting module |
| title | Multi‐stage image inpainting using improved partial convolutions |
| title_full | Multi‐stage image inpainting using improved partial convolutions |
| title_fullStr | Multi‐stage image inpainting using improved partial convolutions |
| title_full_unstemmed | Multi‐stage image inpainting using improved partial convolutions |
| title_short | Multi‐stage image inpainting using improved partial convolutions |
| title_sort | multi stage image inpainting using improved partial convolutions |
| topic | image inpainting image processing image reconstruction multi‐step training process partial inpainting module progressive inpainting module |
| url | https://doi.org/10.1049/ipr2.13178 |
| work_keys_str_mv | AT chengli multistageimageinpaintingusingimprovedpartialconvolutions AT danxu multistageimageinpaintingusingimprovedpartialconvolutions AT haozhang multistageimageinpaintingusingimprovedpartialconvolutions |