Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera
In this paper, we present a vision-based pick-and-place control system for industrial robots using an eye-in-hand camera. In industry, using robots with cameras greatly improves efficiency and performance. Previous studies have focused on utilizing robotic arms for the pick-and-place process in simu...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10858145/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823857115375075328 |
|---|---|
| author | van-Truong Nguyen Phu-Tuan Nguyen Shun-Feng Su Phan Xuan Tan Thanh-Lam Bui |
| author_facet | van-Truong Nguyen Phu-Tuan Nguyen Shun-Feng Su Phan Xuan Tan Thanh-Lam Bui |
| author_sort | van-Truong Nguyen |
| collection | DOAJ |
| description | In this paper, we present a vision-based pick-and-place control system for industrial robots using an eye-in-hand camera. In industry, using robots with cameras greatly improves efficiency and performance. Previous studies have focused on utilizing robotic arms for the pick-and-place process in simulated environments. The challenge when experimenting with real systems lies in aligning the coordinate systems between the robot and the camera, as well as ensuring high data accuracy during experimentation. To address this issue, our research focuses on utilizing a low-cost 2D camera combined with deep learning algorithms mounted on the end-effector of the robotic arm. This study is evaluated in both simulation and real-world experiments. We propose a novel approach that combines the YOLOv7 (You Only Look Once V7) deep learning network with GAN (Generative Adversarial Networks) to achieve fast and accurate object recognition. This system uses deep learning to process camera data to extract object positions for the robot in real-time. Due to its advantages of fast inference and high accuracy, YOLO is applied as the baseline for research. By training the deep learning model on diverse objects, it effectively recognizes and detects any object in the robot’s workspace. Through experimental results, we demonstrate the feasibility and effectiveness of our vision-based pick-and-place system. Our research contributes an important advancement in the field of industrial robots by showcasing the potential of using a 2D camera and an integrated deep learning system for object manipulation. |
| format | Article |
| id | doaj-art-1e9691aae5f4489ebafdd549c0331025 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-1e9691aae5f4489ebafdd549c03310252025-02-12T00:02:01ZengIEEEIEEE Access2169-35362025-01-0113251272514010.1109/ACCESS.2025.353649610858145Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Cameravan-Truong Nguyen0https://orcid.org/0000-0001-6464-2115Phu-Tuan Nguyen1Shun-Feng Su2https://orcid.org/0000-0001-9777-128XPhan Xuan Tan3https://orcid.org/0000-0002-9592-0226Thanh-Lam Bui4Faculty of Mechatronics, SMAE, Hanoi University of Industry, Hanoi, VietnamFaculty of Mechatronics, SMAE, Hanoi University of Industry, Hanoi, VietnamDepartment of Electrical Engineering, National Taiwan University of Science and Technology, Taipei, TaiwanCollege of Engineering, Shibaura Institute of Technology, Tokyo, JapanFaculty of Mechatronics, SMAE, Hanoi University of Industry, Hanoi, VietnamIn this paper, we present a vision-based pick-and-place control system for industrial robots using an eye-in-hand camera. In industry, using robots with cameras greatly improves efficiency and performance. Previous studies have focused on utilizing robotic arms for the pick-and-place process in simulated environments. The challenge when experimenting with real systems lies in aligning the coordinate systems between the robot and the camera, as well as ensuring high data accuracy during experimentation. To address this issue, our research focuses on utilizing a low-cost 2D camera combined with deep learning algorithms mounted on the end-effector of the robotic arm. This study is evaluated in both simulation and real-world experiments. We propose a novel approach that combines the YOLOv7 (You Only Look Once V7) deep learning network with GAN (Generative Adversarial Networks) to achieve fast and accurate object recognition. This system uses deep learning to process camera data to extract object positions for the robot in real-time. Due to its advantages of fast inference and high accuracy, YOLO is applied as the baseline for research. By training the deep learning model on diverse objects, it effectively recognizes and detects any object in the robot’s workspace. Through experimental results, we demonstrate the feasibility and effectiveness of our vision-based pick-and-place system. Our research contributes an important advancement in the field of industrial robots by showcasing the potential of using a 2D camera and an integrated deep learning system for object manipulation.https://ieeexplore.ieee.org/document/10858145/Robotic armvisionobject detectioncalibration visionrobot real-time |
| spellingShingle | van-Truong Nguyen Phu-Tuan Nguyen Shun-Feng Su Phan Xuan Tan Thanh-Lam Bui Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera IEEE Access Robotic arm vision object detection calibration vision robot real-time |
| title | Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera |
| title_full | Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera |
| title_fullStr | Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera |
| title_full_unstemmed | Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera |
| title_short | Vision-Based Pick and Place Control System for Industrial Robots Using an Eye-in-Hand Camera |
| title_sort | vision based pick and place control system for industrial robots using an eye in hand camera |
| topic | Robotic arm vision object detection calibration vision robot real-time |
| url | https://ieeexplore.ieee.org/document/10858145/ |
| work_keys_str_mv | AT vantruongnguyen visionbasedpickandplacecontrolsystemforindustrialrobotsusinganeyeinhandcamera AT phutuannguyen visionbasedpickandplacecontrolsystemforindustrialrobotsusinganeyeinhandcamera AT shunfengsu visionbasedpickandplacecontrolsystemforindustrialrobotsusinganeyeinhandcamera AT phanxuantan visionbasedpickandplacecontrolsystemforindustrialrobotsusinganeyeinhandcamera AT thanhlambui visionbasedpickandplacecontrolsystemforindustrialrobotsusinganeyeinhandcamera |