Semantic‐aware visual consistency network for fused image harmonisation
Abstract With a focus on integrated sensing, communication, and computation (ISCC) systems, multiple sensor devices collect information of different objects and upload it to data processing servers for fusion. Appearance gaps in composite images caused by distinct capture conditions can degrade the...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-06-01
|
| Series: | IET Signal Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/sil2.12219 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832547370356703232 |
|---|---|
| author | Huayan Yu Hai Huang Yueyan Zhu Aoran Chen |
| author_facet | Huayan Yu Hai Huang Yueyan Zhu Aoran Chen |
| author_sort | Huayan Yu |
| collection | DOAJ |
| description | Abstract With a focus on integrated sensing, communication, and computation (ISCC) systems, multiple sensor devices collect information of different objects and upload it to data processing servers for fusion. Appearance gaps in composite images caused by distinct capture conditions can degrade the visual quality and affect the accuracy of other image processing and analysis results. The authors propose a fused‐image harmonisation method that aims to eliminate appearance gaps among different objects. First, the authors modify a lightweight image harmonisation backbone and combined it with a pretrained segmentation model, in which the extracted semantic features were fed to both the encoder and decoder. Then the authors implement a semantic‐related background‐to‐foreground style transfer by leveraging spatial separation adaptive instance normalisation (SAIN). To better preserve the input semantic information, the authors design a simple and effective semantic‐aware adaptive denormalisation (SADE) module. Experimental results demonstrate that the authors’ proposed method achieves competitive performance on the iHarmony4 dataset and benefits from the harmonisation of fused images with incompatible appearance gaps. |
| format | Article |
| id | doaj-art-7f47e658052143deaf70a62c4dbe021d |
| institution | Kabale University |
| issn | 1751-9675 1751-9683 |
| language | English |
| publishDate | 2023-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Signal Processing |
| spelling | doaj-art-7f47e658052143deaf70a62c4dbe021d2025-02-03T06:45:05ZengWileyIET Signal Processing1751-96751751-96832023-06-01176n/an/a10.1049/sil2.12219Semantic‐aware visual consistency network for fused image harmonisationHuayan Yu0Hai Huang1Yueyan Zhu2Aoran Chen3School of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing ChinaSchool of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing ChinaSchool of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing ChinaSchool of Information and Communication Engineering Beijing University of Posts and Telecommunications Beijing ChinaAbstract With a focus on integrated sensing, communication, and computation (ISCC) systems, multiple sensor devices collect information of different objects and upload it to data processing servers for fusion. Appearance gaps in composite images caused by distinct capture conditions can degrade the visual quality and affect the accuracy of other image processing and analysis results. The authors propose a fused‐image harmonisation method that aims to eliminate appearance gaps among different objects. First, the authors modify a lightweight image harmonisation backbone and combined it with a pretrained segmentation model, in which the extracted semantic features were fed to both the encoder and decoder. Then the authors implement a semantic‐related background‐to‐foreground style transfer by leveraging spatial separation adaptive instance normalisation (SAIN). To better preserve the input semantic information, the authors design a simple and effective semantic‐aware adaptive denormalisation (SADE) module. Experimental results demonstrate that the authors’ proposed method achieves competitive performance on the iHarmony4 dataset and benefits from the harmonisation of fused images with incompatible appearance gaps.https://doi.org/10.1049/sil2.12219computer visionimage processing |
| spellingShingle | Huayan Yu Hai Huang Yueyan Zhu Aoran Chen Semantic‐aware visual consistency network for fused image harmonisation IET Signal Processing computer vision image processing |
| title | Semantic‐aware visual consistency network for fused image harmonisation |
| title_full | Semantic‐aware visual consistency network for fused image harmonisation |
| title_fullStr | Semantic‐aware visual consistency network for fused image harmonisation |
| title_full_unstemmed | Semantic‐aware visual consistency network for fused image harmonisation |
| title_short | Semantic‐aware visual consistency network for fused image harmonisation |
| title_sort | semantic aware visual consistency network for fused image harmonisation |
| topic | computer vision image processing |
| url | https://doi.org/10.1049/sil2.12219 |
| work_keys_str_mv | AT huayanyu semanticawarevisualconsistencynetworkforfusedimageharmonisation AT haihuang semanticawarevisualconsistencynetworkforfusedimageharmonisation AT yueyanzhu semanticawarevisualconsistencynetworkforfusedimageharmonisation AT aoranchen semanticawarevisualconsistencynetworkforfusedimageharmonisation |