Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots
Detecting defects in complex urban sewer scenes is crucial for urban underground structure health monitoring. However, most image-based sewer defect detection models are complex, have high resource consumption, and fail to provide detailed damage information. To increase defect detection efficiency,...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7557 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850060324075470848 |
|---|---|
| author | Ruihao Liu Zhongxi Shao Qiang Sun Zhenzhong Yu |
| author_facet | Ruihao Liu Zhongxi Shao Qiang Sun Zhenzhong Yu |
| author_sort | Ruihao Liu |
| collection | DOAJ |
| description | Detecting defects in complex urban sewer scenes is crucial for urban underground structure health monitoring. However, most image-based sewer defect detection models are complex, have high resource consumption, and fail to provide detailed damage information. To increase defect detection efficiency, visualize pipelines, and enable deployment on edge devices, this paper proposes a computer vision-based robotic defect detection framework for sewers. The framework encompasses positioning, defect detection, model deployment, 3D reconstruction, and the measurement of realistic pipelines. A lightweight Sewer-YOLO-Slim model is introduced, which reconstructs the YOLOv7-tiny network by adjusting its backbone, neck, and head. Channel pruning is applied to further reduce the model’s complexity. Additionally, a multiview reconstruction technique is employed to build a 3D model of the pipeline from images captured by the sewer robot, allowing for accurate measurements. The Sewer-YOLO-Slim model achieves reductions of 60.2%, 60.0%, and 65.9% in model size, parameters, and floating-point operations (FLOPs), respectively, while improving the mean average precision (mAP) by 1.5%, reaching 93.5%. Notably, the pruned model is only 4.9 MB in size. Comprehensive comparisons and analyses are conducted with 12 mainstream detection algorithms to validate the superiority of the proposed model. The model is deployed on edge devices with the aid of TensorRT for acceleration, and the detection speed reaches 15.3 ms per image. For a real section of the pipeline, the maximum measurement error of the 3D reconstruction model is 0.57 m. These results indicate that the proposed sewer inspection framework is effective, with the detection model exhibiting advanced performance in terms of accuracy, low computational demand, and real-time capability. The 3D modeling approach offers valuable insights for underground pipeline data visualization and defect measurement. |
| format | Article |
| id | doaj-art-14ae1df7e8854d6c803dcc2f393ac86e |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-14ae1df7e8854d6c803dcc2f393ac86e2025-08-20T02:50:36ZengMDPI AGSensors1424-82202024-11-012423755710.3390/s24237557Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer RobotsRuihao Liu0Zhongxi Shao1Qiang Sun2Zhenzhong Yu3School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaSchool of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150001, ChinaHefei Intelligent Robot Institute, Hefei 230601, ChinaHefei Intelligent Robot Institute, Hefei 230601, ChinaDetecting defects in complex urban sewer scenes is crucial for urban underground structure health monitoring. However, most image-based sewer defect detection models are complex, have high resource consumption, and fail to provide detailed damage information. To increase defect detection efficiency, visualize pipelines, and enable deployment on edge devices, this paper proposes a computer vision-based robotic defect detection framework for sewers. The framework encompasses positioning, defect detection, model deployment, 3D reconstruction, and the measurement of realistic pipelines. A lightweight Sewer-YOLO-Slim model is introduced, which reconstructs the YOLOv7-tiny network by adjusting its backbone, neck, and head. Channel pruning is applied to further reduce the model’s complexity. Additionally, a multiview reconstruction technique is employed to build a 3D model of the pipeline from images captured by the sewer robot, allowing for accurate measurements. The Sewer-YOLO-Slim model achieves reductions of 60.2%, 60.0%, and 65.9% in model size, parameters, and floating-point operations (FLOPs), respectively, while improving the mean average precision (mAP) by 1.5%, reaching 93.5%. Notably, the pruned model is only 4.9 MB in size. Comprehensive comparisons and analyses are conducted with 12 mainstream detection algorithms to validate the superiority of the proposed model. The model is deployed on edge devices with the aid of TensorRT for acceleration, and the detection speed reaches 15.3 ms per image. For a real section of the pipeline, the maximum measurement error of the 3D reconstruction model is 0.57 m. These results indicate that the proposed sewer inspection framework is effective, with the detection model exhibiting advanced performance in terms of accuracy, low computational demand, and real-time capability. The 3D modeling approach offers valuable insights for underground pipeline data visualization and defect measurement.https://www.mdpi.com/1424-8220/24/23/7557urban pipelinedefect detectionYOLO networklightweightedge deployment3D reconstruction |
| spellingShingle | Ruihao Liu Zhongxi Shao Qiang Sun Zhenzhong Yu Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots Sensors urban pipeline defect detection YOLO network lightweight edge deployment 3D reconstruction |
| title | Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots |
| title_full | Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots |
| title_fullStr | Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots |
| title_full_unstemmed | Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots |
| title_short | Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots |
| title_sort | defect detection and 3d reconstruction of complex urban underground pipeline scenes for sewer robots |
| topic | urban pipeline defect detection YOLO network lightweight edge deployment 3D reconstruction |
| url | https://www.mdpi.com/1424-8220/24/23/7557 |
| work_keys_str_mv | AT ruihaoliu defectdetectionand3dreconstructionofcomplexurbanundergroundpipelinescenesforsewerrobots AT zhongxishao defectdetectionand3dreconstructionofcomplexurbanundergroundpipelinescenesforsewerrobots AT qiangsun defectdetectionand3dreconstructionofcomplexurbanundergroundpipelinescenesforsewerrobots AT zhenzhongyu defectdetectionand3dreconstructionofcomplexurbanundergroundpipelinescenesforsewerrobots |