Pose measurement method for coal mine drilling robot based on deep learning
Underground drilling is a commonly used construction method that requires repeated loading and unloading of drill pipes. To achieve intelligent loading and unloading of drill pipes, the measurement of the drilling position is particularly important. However, the complex environment in underground co...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Office of Journal of China Coal Society
2025-07-01
|
| Series: | Meitan xuebao |
| Subjects: | |
| Online Access: | http://www.mtxb.com.cn/article/doi/10.13225/j.cnki.jccs.2024.0831 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849772563491717120 |
|---|---|
| author | Jiangnan LUO Jianping LI Hongxiang JIANG Deyi ZHANG |
| author_facet | Jiangnan LUO Jianping LI Hongxiang JIANG Deyi ZHANG |
| author_sort | Jiangnan LUO |
| collection | DOAJ |
| description | Underground drilling is a commonly used construction method that requires repeated loading and unloading of drill pipes. To achieve intelligent loading and unloading of drill pipes, the measurement of the drilling position is particularly important. However, the complex environment in underground coal mines, characterized by dust, fog, and variable lighting conditions, makes traditional recognition methods inadequate. Additionally, since the drill rod is not installed on the drilling rig during delivery, direct measurement is impractical. To address the challenge of measuring the drilling position in underground coal mines, a deep learning-based method is proposed. This method consists of two parts: a segmentation model based on an improved PointNet++ and a point cloud registration process using the drill head and gripper. First, to address the current insufficiency of publicly available drilling rig point cloud datasets, we have established a platform for acquiring point cloud data from drilling rigs. A 3D camera is utilized to capture point cloud data at night, with illumination solely provided by LED lights to simulate the uneven lighting conditions found underground. To simulate the dusty and foggy underground environment, salt-and-pepper noise was added to the point cloud data, and noise was separately classified when creating labels to achieve a denoising effect. Next, a generative adversarial network was integrated into the PointNet++ to capture more complex and detailed point cloud features. A focal loss function was employed to enhance the model's focus on the drill head and gripper, and Bayesian parameter optimization was used for hyperparameter tuning. Then, the measured point cloud is registered using the Fast Point Feature Histogram (FPFH) and Iterative Closest Point (ICP) algorithms to obtain the transformation matrix from the measured point cloud to the source point cloud. Finally, the drilling position and direction vectors were determined, thereby defining the drilling position. To evaluate the measurement accuracy of the proposed method, six sets of point cloud data of the drilling scene with the drill pipe mounted on the rig were collected. The drill pipe’s position and orientation were then measured by manually segmenting the data using Cloud Compare software. Experimental results on the self-built dataset show that the improved PointNet++ model achieved a 17.7% and 37.8% improvement in Intersection over Union (IoU) and Precision, respectively. Specifically, the IoU values for the drill head and gripper increased by 34.9% and 60.3%, respectively. In terms of drilling position measurement, the average distance error was 6.39 mm, the radial distance error was 5.34 mm, and the average angle error was 1.6°. Therefore, the proposed measurement method for drill pipe delivery posture is feasible and has potential application value in the field of intelligent coal mine drill pipe loading and unloading. |
| format | Article |
| id | doaj-art-04ebaf87340b4c77b55b6695e96ffdf4 |
| institution | DOAJ |
| issn | 0253-9993 |
| language | zho |
| publishDate | 2025-07-01 |
| publisher | Editorial Office of Journal of China Coal Society |
| record_format | Article |
| series | Meitan xuebao |
| spelling | doaj-art-04ebaf87340b4c77b55b6695e96ffdf42025-08-20T03:02:18ZzhoEditorial Office of Journal of China Coal SocietyMeitan xuebao0253-99932025-07-015073679369110.13225/j.cnki.jccs.2024.08312024-0831Pose measurement method for coal mine drilling robot based on deep learningJiangnan LUO0Jianping LI1Hongxiang JIANG2Deyi ZHANG3School of Mechatronic and Electrical Engineering, China University of Mining and Technology, Xuzhou 221116, ChinaSchool of Mechatronic and Electrical Engineering, China University of Mining and Technology, Xuzhou 221116, ChinaSchool of Mechatronic and Electrical Engineering, China University of Mining and Technology, Xuzhou 221116, ChinaSchool of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaUnderground drilling is a commonly used construction method that requires repeated loading and unloading of drill pipes. To achieve intelligent loading and unloading of drill pipes, the measurement of the drilling position is particularly important. However, the complex environment in underground coal mines, characterized by dust, fog, and variable lighting conditions, makes traditional recognition methods inadequate. Additionally, since the drill rod is not installed on the drilling rig during delivery, direct measurement is impractical. To address the challenge of measuring the drilling position in underground coal mines, a deep learning-based method is proposed. This method consists of two parts: a segmentation model based on an improved PointNet++ and a point cloud registration process using the drill head and gripper. First, to address the current insufficiency of publicly available drilling rig point cloud datasets, we have established a platform for acquiring point cloud data from drilling rigs. A 3D camera is utilized to capture point cloud data at night, with illumination solely provided by LED lights to simulate the uneven lighting conditions found underground. To simulate the dusty and foggy underground environment, salt-and-pepper noise was added to the point cloud data, and noise was separately classified when creating labels to achieve a denoising effect. Next, a generative adversarial network was integrated into the PointNet++ to capture more complex and detailed point cloud features. A focal loss function was employed to enhance the model's focus on the drill head and gripper, and Bayesian parameter optimization was used for hyperparameter tuning. Then, the measured point cloud is registered using the Fast Point Feature Histogram (FPFH) and Iterative Closest Point (ICP) algorithms to obtain the transformation matrix from the measured point cloud to the source point cloud. Finally, the drilling position and direction vectors were determined, thereby defining the drilling position. To evaluate the measurement accuracy of the proposed method, six sets of point cloud data of the drilling scene with the drill pipe mounted on the rig were collected. The drill pipe’s position and orientation were then measured by manually segmenting the data using Cloud Compare software. Experimental results on the self-built dataset show that the improved PointNet++ model achieved a 17.7% and 37.8% improvement in Intersection over Union (IoU) and Precision, respectively. Specifically, the IoU values for the drill head and gripper increased by 34.9% and 60.3%, respectively. In terms of drilling position measurement, the average distance error was 6.39 mm, the radial distance error was 5.34 mm, and the average angle error was 1.6°. Therefore, the proposed measurement method for drill pipe delivery posture is feasible and has potential application value in the field of intelligent coal mine drill pipe loading and unloading.http://www.mtxb.com.cn/article/doi/10.13225/j.cnki.jccs.2024.0831drill pipe installation pose3d visionpoint cloud segmentationdeep learninggenerative adversarial networkdrilling intelligence |
| spellingShingle | Jiangnan LUO Jianping LI Hongxiang JIANG Deyi ZHANG Pose measurement method for coal mine drilling robot based on deep learning Meitan xuebao drill pipe installation pose 3d vision point cloud segmentation deep learning generative adversarial network drilling intelligence |
| title | Pose measurement method for coal mine drilling robot based on deep learning |
| title_full | Pose measurement method for coal mine drilling robot based on deep learning |
| title_fullStr | Pose measurement method for coal mine drilling robot based on deep learning |
| title_full_unstemmed | Pose measurement method for coal mine drilling robot based on deep learning |
| title_short | Pose measurement method for coal mine drilling robot based on deep learning |
| title_sort | pose measurement method for coal mine drilling robot based on deep learning |
| topic | drill pipe installation pose 3d vision point cloud segmentation deep learning generative adversarial network drilling intelligence |
| url | http://www.mtxb.com.cn/article/doi/10.13225/j.cnki.jccs.2024.0831 |
| work_keys_str_mv | AT jiangnanluo posemeasurementmethodforcoalminedrillingrobotbasedondeeplearning AT jianpingli posemeasurementmethodforcoalminedrillingrobotbasedondeeplearning AT hongxiangjiang posemeasurementmethodforcoalminedrillingrobotbasedondeeplearning AT deyizhang posemeasurementmethodforcoalminedrillingrobotbasedondeeplearning |