Explainable analysis of infrared and visible light image fusion based on deep learning
Abstract Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimoda...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-79684-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832594808586108928 |
|---|---|
| author | Bo Yuan Hongyu Sun YinJing Guo Qiang Liu Xinghao Zhan |
| author_facet | Bo Yuan Hongyu Sun YinJing Guo Qiang Liu Xinghao Zhan |
| author_sort | Bo Yuan |
| collection | DOAJ |
| description | Abstract Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image fusion model was proposed based on the advantages of convolutional neural networks (CNN) for local context extraction and Transformer global attention mechanism. Secondly, to enhance the explainability of the model, the Delta Debugging Fuse Image (DDFImage) algorithm was employed for generating local explanatory information. Finally, we gain deeper insights into the internal workings of the model through feature importance analysis of the generated explanatory fusion images. Comparative analysis with other explainability algorithms demonstrates the superior performance of our algorithm. This comprehensive approach not only improves the explainability of the model but also provides more reference for practical application of the model. |
| format | Article |
| id | doaj-art-0d2f57c12db64738a71d0275231916d4 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-0d2f57c12db64738a71d0275231916d42025-01-19T12:19:22ZengNature PortfolioScientific Reports2045-23222025-01-0115111010.1038/s41598-024-79684-6Explainable analysis of infrared and visible light image fusion based on deep learningBo Yuan0Hongyu Sun1YinJing Guo2Qiang Liu3Xinghao Zhan4Shandong University of Science and TechnologyShandong University of Science and TechnologyShandong University of Science and TechnologyShandong University of Science and TechnologyShandong University of Science and TechnologyAbstract Explainability is a very active area of research in machine learning and image processing. This paper aims to investigate the explainability of visible light and infrared image fusion technology in order to enhance the credibility of model understanding and application. Firstly, a multimodal image fusion model was proposed based on the advantages of convolutional neural networks (CNN) for local context extraction and Transformer global attention mechanism. Secondly, to enhance the explainability of the model, the Delta Debugging Fuse Image (DDFImage) algorithm was employed for generating local explanatory information. Finally, we gain deeper insights into the internal workings of the model through feature importance analysis of the generated explanatory fusion images. Comparative analysis with other explainability algorithms demonstrates the superior performance of our algorithm. This comprehensive approach not only improves the explainability of the model but also provides more reference for practical application of the model.https://doi.org/10.1038/s41598-024-79684-6Machine learningExplainabilityImage fusionLocal explanations |
| spellingShingle | Bo Yuan Hongyu Sun YinJing Guo Qiang Liu Xinghao Zhan Explainable analysis of infrared and visible light image fusion based on deep learning Scientific Reports Machine learning Explainability Image fusion Local explanations |
| title | Explainable analysis of infrared and visible light image fusion based on deep learning |
| title_full | Explainable analysis of infrared and visible light image fusion based on deep learning |
| title_fullStr | Explainable analysis of infrared and visible light image fusion based on deep learning |
| title_full_unstemmed | Explainable analysis of infrared and visible light image fusion based on deep learning |
| title_short | Explainable analysis of infrared and visible light image fusion based on deep learning |
| title_sort | explainable analysis of infrared and visible light image fusion based on deep learning |
| topic | Machine learning Explainability Image fusion Local explanations |
| url | https://doi.org/10.1038/s41598-024-79684-6 |
| work_keys_str_mv | AT boyuan explainableanalysisofinfraredandvisiblelightimagefusionbasedondeeplearning AT hongyusun explainableanalysisofinfraredandvisiblelightimagefusionbasedondeeplearning AT yinjingguo explainableanalysisofinfraredandvisiblelightimagefusionbasedondeeplearning AT qiangliu explainableanalysisofinfraredandvisiblelightimagefusionbasedondeeplearning AT xinghaozhan explainableanalysisofinfraredandvisiblelightimagefusionbasedondeeplearning |