Pseudolabel guided pixels contrast for domain adaptive semantic segmentation
Abstract Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level. Acquiring such annotations can be costly in the real-world. Unsupervised domain adaptation (UDA) for semantic segmentation is a techniqu...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-78404-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559482823344128 |
|---|---|
| author | Jianzi Xiang Cailu Wan Zhu Cao |
| author_facet | Jianzi Xiang Cailu Wan Zhu Cao |
| author_sort | Jianzi Xiang |
| collection | DOAJ |
| description | Abstract Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level. Acquiring such annotations can be costly in the real-world. Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. However, these works do not take into account the diversity of features within each class when using contrastive learning, which leads to errors in class prediction. We analyze the limitations of these works and propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods. We also investigate how to use more information from target images without adding noise from pseudo-labels. We test our method on two standard UDA benchmarks and show that it outperforms existing methods. Specifically, we achieve relative improvements of 5.1% mIoU and 4.6% mIoU on the Grand Theft Auto V (GTA5) to Cityscapes and SYNTHIA to Cityscapes tasks based on DAFormer, respectively. Furthermore, our approach can enhance the performance of other UDA approaches without increasing model complexity. |
| format | Article |
| id | doaj-art-09b6a5333fa5481b8fa89b1fdd334619 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-09b6a5333fa5481b8fa89b1fdd3346192025-01-05T12:25:41ZengNature PortfolioScientific Reports2045-23222024-12-0114111810.1038/s41598-024-78404-4Pseudolabel guided pixels contrast for domain adaptive semantic segmentationJianzi Xiang0Cailu Wan1Zhu Cao2The Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and TechnologyThe Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and TechnologyThe Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and TechnologyAbstract Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level. Acquiring such annotations can be costly in the real-world. Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. However, these works do not take into account the diversity of features within each class when using contrastive learning, which leads to errors in class prediction. We analyze the limitations of these works and propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods. We also investigate how to use more information from target images without adding noise from pseudo-labels. We test our method on two standard UDA benchmarks and show that it outperforms existing methods. Specifically, we achieve relative improvements of 5.1% mIoU and 4.6% mIoU on the Grand Theft Auto V (GTA5) to Cityscapes and SYNTHIA to Cityscapes tasks based on DAFormer, respectively. Furthermore, our approach can enhance the performance of other UDA approaches without increasing model complexity.https://doi.org/10.1038/s41598-024-78404-4Semantic segmentationUnsupervised domain adaptationContrastive learning |
| spellingShingle | Jianzi Xiang Cailu Wan Zhu Cao Pseudolabel guided pixels contrast for domain adaptive semantic segmentation Scientific Reports Semantic segmentation Unsupervised domain adaptation Contrastive learning |
| title | Pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| title_full | Pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| title_fullStr | Pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| title_full_unstemmed | Pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| title_short | Pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| title_sort | pseudolabel guided pixels contrast for domain adaptive semantic segmentation |
| topic | Semantic segmentation Unsupervised domain adaptation Contrastive learning |
| url | https://doi.org/10.1038/s41598-024-78404-4 |
| work_keys_str_mv | AT jianzixiang pseudolabelguidedpixelscontrastfordomainadaptivesemanticsegmentation AT cailuwan pseudolabelguidedpixelscontrastfordomainadaptivesemanticsegmentation AT zhucao pseudolabelguidedpixelscontrastfordomainadaptivesemanticsegmentation |