Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) serves as a potential alternative for improving cross- domain segmentation tasks. Recent UDA approaches have identified class-wise prototypes and leveraged them to guide the segmentation process in the target domain. However, these methods overlook additional inf...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11007591/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850132610029715456 |
|---|---|
| author | Antonio Dauphin Fernando Thumma Anirudh Selvaraj Palanisamy Karthika Prasad Katia Alexander Pandiyarasan Veluswamy Rohini Palanisamy |
| author_facet | Antonio Dauphin Fernando Thumma Anirudh Selvaraj Palanisamy Karthika Prasad Katia Alexander Pandiyarasan Veluswamy Rohini Palanisamy |
| author_sort | Antonio Dauphin Fernando |
| collection | DOAJ |
| description | Unsupervised Domain Adaptation (UDA) serves as a potential alternative for improving cross- domain segmentation tasks. Recent UDA approaches have identified class-wise prototypes and leveraged them to guide the segmentation process in the target domain. However, these methods overlook additional information from other auxiliary tasks, such as depth, which can potentially improve overall segmentation performance. This paper proposes a Depth-Aware Prototypical learning for Semantic Segmentation (DA- ProSS) pipeline, which includes a novel multitask prototype learning framework that comprises task-specific and task-integrated classifiers. The task-specific classifiers capture the semantic and depth features, and the task-integrated classifier captures the hidden semantic features from depth prediction. This framework ensures that the prototypes from respective heads learn better class-representative features using semantic information and depth cues. Additionally, this pipeline integrates a Self-Refinement learning (SRL) algorithm that generates cross-domain pseudo-labels, which are leveraged to generate refined targets for further self-supervised training. Results indicate that prototypes generated through the depth-encoded semantic task could understand the underlying semantics of the object. The proposed DA-ProSS pipeline with SRL helps the model generalize better to achieve a mIoU of 57.3 that outperforms the previous state-of-the-art methods in the SYNTHIA-to-Cityscapes benchmark dataset. |
| format | Article |
| id | doaj-art-d31b59c27be44520879b70c0164f3f95 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d31b59c27be44520879b70c0164f3f952025-08-20T02:32:10ZengIEEEIEEE Access2169-35362025-01-0113957069571610.1109/ACCESS.2025.357201311007591Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain AdaptationAntonio Dauphin Fernando0Thumma Anirudh1Selvaraj Palanisamy2Karthika Prasad3Katia Alexander4Pandiyarasan Veluswamy5Rohini Palanisamy6https://orcid.org/0000-0001-7050-7681Department of Electronics and Communication Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, Chennai, IndiaDepartment of Electronics and Communication Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, Chennai, IndiaDepartment of Clinical Medicine, Tamil Nadu Veterinary and Animal Sciences University, Chennai, IndiaSchool of Engineering, College of Systems and Society, The Australian National University, Canberra, ACT, AustraliaSchool of Engineering, College of Systems and Society, The Australian National University, Canberra, ACT, AustraliaDepartment of Electronics and Communication Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, Chennai, IndiaDepartment of Electronics and Communication Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, Chennai, IndiaUnsupervised Domain Adaptation (UDA) serves as a potential alternative for improving cross- domain segmentation tasks. Recent UDA approaches have identified class-wise prototypes and leveraged them to guide the segmentation process in the target domain. However, these methods overlook additional information from other auxiliary tasks, such as depth, which can potentially improve overall segmentation performance. This paper proposes a Depth-Aware Prototypical learning for Semantic Segmentation (DA- ProSS) pipeline, which includes a novel multitask prototype learning framework that comprises task-specific and task-integrated classifiers. The task-specific classifiers capture the semantic and depth features, and the task-integrated classifier captures the hidden semantic features from depth prediction. This framework ensures that the prototypes from respective heads learn better class-representative features using semantic information and depth cues. Additionally, this pipeline integrates a Self-Refinement learning (SRL) algorithm that generates cross-domain pseudo-labels, which are leveraged to generate refined targets for further self-supervised training. Results indicate that prototypes generated through the depth-encoded semantic task could understand the underlying semantics of the object. The proposed DA-ProSS pipeline with SRL helps the model generalize better to achieve a mIoU of 57.3 that outperforms the previous state-of-the-art methods in the SYNTHIA-to-Cityscapes benchmark dataset.https://ieeexplore.ieee.org/document/11007591/Contrastive learningprototype learningself-refinement learningunsupervised domain adaptationdepth integration |
| spellingShingle | Antonio Dauphin Fernando Thumma Anirudh Selvaraj Palanisamy Karthika Prasad Katia Alexander Pandiyarasan Veluswamy Rohini Palanisamy Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation IEEE Access Contrastive learning prototype learning self-refinement learning unsupervised domain adaptation depth integration |
| title | Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation |
| title_full | Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation |
| title_fullStr | Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation |
| title_full_unstemmed | Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation |
| title_short | Depth Integrated Multi-Task Prototypical Learning With Self Refinement for Unsupervised Domain Adaptation |
| title_sort | depth integrated multi task prototypical learning with self refinement for unsupervised domain adaptation |
| topic | Contrastive learning prototype learning self-refinement learning unsupervised domain adaptation depth integration |
| url | https://ieeexplore.ieee.org/document/11007591/ |
| work_keys_str_mv | AT antoniodauphinfernando depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT thummaanirudh depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT selvarajpalanisamy depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT karthikaprasad depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT katiaalexander depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT pandiyarasanveluswamy depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation AT rohinipalanisamy depthintegratedmultitaskprototypicallearningwithselfrefinementforunsuperviseddomainadaptation |