MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation
Accurate polyp image segmentation is of great significance, because it can help in the detection of polyps. Convolutional neural network (CNN) is a common automatic segmentation method, but its main disadvantage is the long training time. Transformer is another method that can be adapted to the auto...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7473 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850060213223161856 |
|---|---|
| author | Chen Peng Zhiqin Qian Kunyu Wang Lanzhu Zhang Qi Luo Zhuming Bi Wenjun Zhang |
| author_facet | Chen Peng Zhiqin Qian Kunyu Wang Lanzhu Zhang Qi Luo Zhuming Bi Wenjun Zhang |
| author_sort | Chen Peng |
| collection | DOAJ |
| description | Accurate polyp image segmentation is of great significance, because it can help in the detection of polyps. Convolutional neural network (CNN) is a common automatic segmentation method, but its main disadvantage is the long training time. Transformer is another method that can be adapted to the automatic segmentation method by employing a self-attention mechanism, which essentially assigns different importance weights to each piece of information, thus achieving high computational efficiency during segmentation. However, a potential drawback with Transformer is the risk of information loss. The study reported in this paper employed the well-known hybridization principle to propose a method to combine CNN and Transformer to retain the strengths of both. Specifically, this study applied this method to the early detection of colonic polyps and to implement a model called MugenNet for colonic polyp image segmentation. We conducted a comprehensive experiment to compare MugenNet with other CNN models on five publicly available datasets. An ablation experiment on MugenNet was conducted as well. The experimental results showed that MugenNet can achieve a mean Dice of 0.714 on the ETIS dataset, which is the optimal performance on this dataset compared to other models, with an inference speed of 56 FPS. The overall outcome of this study is a method to optimally combine two methods of machine learning which are complementary to each other. |
| format | Article |
| id | doaj-art-5823d87f16d742a6a8982d5b3b3e7f08 |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-5823d87f16d742a6a8982d5b3b3e7f082025-08-20T02:50:40ZengMDPI AGSensors1424-82202024-11-012423747310.3390/s24237473MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image SegmentationChen Peng0Zhiqin Qian1Kunyu Wang2Lanzhu Zhang3Qi Luo4Zhuming Bi5Wenjun Zhang6School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, ChinaSchool of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, ChinaSchool of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, ChinaSchool of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, ChinaSchool of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, ChinaDepartment of Engineering, Purdue University, West Lafayette, IN 47907, USADepartment of Mechanical Engineering, University of Saskatchewan, Saskatoon, SK S7N 5A9, CanadaAccurate polyp image segmentation is of great significance, because it can help in the detection of polyps. Convolutional neural network (CNN) is a common automatic segmentation method, but its main disadvantage is the long training time. Transformer is another method that can be adapted to the automatic segmentation method by employing a self-attention mechanism, which essentially assigns different importance weights to each piece of information, thus achieving high computational efficiency during segmentation. However, a potential drawback with Transformer is the risk of information loss. The study reported in this paper employed the well-known hybridization principle to propose a method to combine CNN and Transformer to retain the strengths of both. Specifically, this study applied this method to the early detection of colonic polyps and to implement a model called MugenNet for colonic polyp image segmentation. We conducted a comprehensive experiment to compare MugenNet with other CNN models on five publicly available datasets. An ablation experiment on MugenNet was conducted as well. The experimental results showed that MugenNet can achieve a mean Dice of 0.714 on the ETIS dataset, which is the optimal performance on this dataset compared to other models, with an inference speed of 56 FPS. The overall outcome of this study is a method to optimally combine two methods of machine learning which are complementary to each other.https://www.mdpi.com/1424-8220/24/23/7473transformerconvolutional neural networkpolyp detectionimage segmentation |
| spellingShingle | Chen Peng Zhiqin Qian Kunyu Wang Lanzhu Zhang Qi Luo Zhuming Bi Wenjun Zhang MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation Sensors transformer convolutional neural network polyp detection image segmentation |
| title | MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation |
| title_full | MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation |
| title_fullStr | MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation |
| title_full_unstemmed | MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation |
| title_short | MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with Application in Colonic Polyp Image Segmentation |
| title_sort | mugennet a novel combined convolution neural network and transformer network with application in colonic polyp image segmentation |
| topic | transformer convolutional neural network polyp detection image segmentation |
| url | https://www.mdpi.com/1424-8220/24/23/7473 |
| work_keys_str_mv | AT chenpeng mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT zhiqinqian mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT kunyuwang mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT lanzhuzhang mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT qiluo mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT zhumingbi mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation AT wenjunzhang mugennetanovelcombinedconvolutionneuralnetworkandtransformernetworkwithapplicationincolonicpolypimagesegmentation |