Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features
In order to solve the problem of the distribution differences of visual, spatial, and semantic features between domains in domain adaptation, an unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features was proposed.Firstly, an attentive f...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2022-12-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2022212/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841540000134463488 |
|---|---|
| author | Zeyu WANG Shuhui BU Wei HUANG Yuanpan ZHENG Qinggang WU Huawen CHANG Xu ZHANG |
| author_facet | Zeyu WANG Shuhui BU Wei HUANG Yuanpan ZHENG Qinggang WU Huawen CHANG Xu ZHANG |
| author_sort | Zeyu WANG |
| collection | DOAJ |
| description | In order to solve the problem of the distribution differences of visual, spatial, and semantic features between domains in domain adaptation, an unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features was proposed.Firstly, an attentive fusion semantic segmentation network with three-layer structure was designed to learn the above three types of features from the source domain and target domain, respectively.Secondly, a self-supervised learning method jointing distribution confidence and semantic confidence was introduced into the single-level adversarial learning, so as to achieve the distribution alignment of more target domain pixels in the process of minimizing the distribution distance of the learnt features between domains.Finally, three adversarial branches and three adaptive sub-networks were jointly optimized by the multi-level adversarial learning method based on multi-modal features, which could effectively learn the invariant representation between domains for the features extracted from each sub-network.The experimental results show that compared with existing state-of-the-art methods, on the datasets of GTA5 to Cityscapes, SYNTHIA to Cityscapes, and SUN-RGBD to NYUD-v2 the proposed network achieves the best mean intersection over union of 62.2%, 66.9%, and 59.7%, respectively. |
| format | Article |
| id | doaj-art-e5234732e9da4e458265e43227f4e683 |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2022-12-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-e5234732e9da4e458265e43227f4e6832025-01-14T06:28:39ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2022-12-014315717159391161Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal featuresZeyu WANGShuhui BUWei HUANGYuanpan ZHENGQinggang WUHuawen CHANGXu ZHANGIn order to solve the problem of the distribution differences of visual, spatial, and semantic features between domains in domain adaptation, an unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features was proposed.Firstly, an attentive fusion semantic segmentation network with three-layer structure was designed to learn the above three types of features from the source domain and target domain, respectively.Secondly, a self-supervised learning method jointing distribution confidence and semantic confidence was introduced into the single-level adversarial learning, so as to achieve the distribution alignment of more target domain pixels in the process of minimizing the distribution distance of the learnt features between domains.Finally, three adversarial branches and three adaptive sub-networks were jointly optimized by the multi-level adversarial learning method based on multi-modal features, which could effectively learn the invariant representation between domains for the features extracted from each sub-network.The experimental results show that compared with existing state-of-the-art methods, on the datasets of GTA5 to Cityscapes, SYNTHIA to Cityscapes, and SUN-RGBD to NYUD-v2 the proposed network achieves the best mean intersection over union of 62.2%, 66.9%, and 59.7%, respectively.http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2022212/unsupervised domain adaptationsemantic segmentationmulti-modal featuresattentive fusionmulti-level adversarial learningself-supervised learning |
| spellingShingle | Zeyu WANG Shuhui BU Wei HUANG Yuanpan ZHENG Qinggang WU Huawen CHANG Xu ZHANG Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features Tongxin xuebao unsupervised domain adaptation semantic segmentation multi-modal features attentive fusion multi-level adversarial learning self-supervised learning |
| title | Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features |
| title_full | Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features |
| title_fullStr | Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features |
| title_full_unstemmed | Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features |
| title_short | Unsupervised domain adaptation multi-level adversarial network for semantic segmentation based on multi-modal features |
| title_sort | unsupervised domain adaptation multi level adversarial network for semantic segmentation based on multi modal features |
| topic | unsupervised domain adaptation semantic segmentation multi-modal features attentive fusion multi-level adversarial learning self-supervised learning |
| url | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2022212/ |
| work_keys_str_mv | AT zeyuwang unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT shuhuibu unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT weihuang unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT yuanpanzheng unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT qinggangwu unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT huawenchang unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures AT xuzhang unsuperviseddomainadaptationmultileveladversarialnetworkforsemanticsegmentationbasedonmultimodalfeatures |