Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly
Robotic assembly is widely utilized in large-scale manufacturing due to its high production efficiency, and the peg-in-hole assembly is a typical operation. While single peg-in-hole tasks have achieved great performance through reinforcement learning (RL) methods, multiple peg-in-hole assembly remai...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-10-01
|
| Series: | CAAI Artificial Intelligence Research |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/AIR.2024.9150043 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550161060298752 |
|---|---|
| author | Xinyu Liu Chao Zeng Chenguang Yang Jianwei Zhang |
| author_facet | Xinyu Liu Chao Zeng Chenguang Yang Jianwei Zhang |
| author_sort | Xinyu Liu |
| collection | DOAJ |
| description | Robotic assembly is widely utilized in large-scale manufacturing due to its high production efficiency, and the peg-in-hole assembly is a typical operation. While single peg-in-hole tasks have achieved great performance through reinforcement learning (RL) methods, multiple peg-in-hole assembly remains challenging due to complex geometry and physical constraints. To address this, we introduce a control policy workflow for multiple peg-in-hole assembly, dividing the task into three primitive sub-tasks: picking, alignment, and insertion to modularize the long-term task and improve sample efficiency. Sequential control policy (SeqPolicy), containing three control policies, is used to implement all the sub-tasks step-by-step. This approach introduces human knowledge to manage intermediate states, such as lifting height and aligning direction, thereby enabling flexible deployment across various scenarios. SeqPolicy demonstrated higher training efficiency with faster convergence and a higher success rate compared to the single control policy. Its adaptability is confirmed through generalization experiments involving objects with varying geometries. Recognizing the importance of object pose for control policies, a low-cost and adaptable method using visual representation containing objects’ pose information from RGB images is proposed to estimate objects’ pose in robot base frame directly in working scenarios. The representation is extracted by a Siamese-CNN network trained with self-supervised contrastive learning. Utilizing it, the alignment sub-task is successfully executed. These experiments validate the solution’s reusability and adaptability in multiple peg-in-hole scenarios. |
| format | Article |
| id | doaj-art-1735c233801b4573a641109d9a06d099 |
| institution | Kabale University |
| issn | 2097-194X 2097-3691 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | CAAI Artificial Intelligence Research |
| spelling | doaj-art-1735c233801b4573a641109d9a06d0992025-01-10T06:44:32ZengTsinghua University PressCAAI Artificial Intelligence Research2097-194X2097-36912024-10-013915004310.26599/AIR.2024.9150043Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole AssemblyXinyu Liu0Chao Zeng1Chenguang Yang2Jianwei Zhang3Department of Electrical and Photonics Engineering, Technical University of Denmark, Kongens Lyngby 2800, DenmarkDepartment of Computer Science, University of Liverpool, Liverpool L69 3BX, UKDepartment of Computer Science, University of Liverpool, Liverpool L69 3BX, UKTAMS Group, Department of Informatics, University of Hamburg, Hamburg 22527, GermanyRobotic assembly is widely utilized in large-scale manufacturing due to its high production efficiency, and the peg-in-hole assembly is a typical operation. While single peg-in-hole tasks have achieved great performance through reinforcement learning (RL) methods, multiple peg-in-hole assembly remains challenging due to complex geometry and physical constraints. To address this, we introduce a control policy workflow for multiple peg-in-hole assembly, dividing the task into three primitive sub-tasks: picking, alignment, and insertion to modularize the long-term task and improve sample efficiency. Sequential control policy (SeqPolicy), containing three control policies, is used to implement all the sub-tasks step-by-step. This approach introduces human knowledge to manage intermediate states, such as lifting height and aligning direction, thereby enabling flexible deployment across various scenarios. SeqPolicy demonstrated higher training efficiency with faster convergence and a higher success rate compared to the single control policy. Its adaptability is confirmed through generalization experiments involving objects with varying geometries. Recognizing the importance of object pose for control policies, a low-cost and adaptable method using visual representation containing objects’ pose information from RGB images is proposed to estimate objects’ pose in robot base frame directly in working scenarios. The representation is extracted by a Siamese-CNN network trained with self-supervised contrastive learning. Utilizing it, the alignment sub-task is successfully executed. These experiments validate the solution’s reusability and adaptability in multiple peg-in-hole scenarios.https://www.sciopen.com/article/10.26599/AIR.2024.9150043multiple peg-in-hole assemblydeep reinforcement learningself-supervised contrastive learning |
| spellingShingle | Xinyu Liu Chao Zeng Chenguang Yang Jianwei Zhang Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly CAAI Artificial Intelligence Research multiple peg-in-hole assembly deep reinforcement learning self-supervised contrastive learning |
| title | Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly |
| title_full | Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly |
| title_fullStr | Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly |
| title_full_unstemmed | Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly |
| title_short | Reinforcement Learning-Based Sequential Control Policy for Multiple Peg-in-Hole Assembly |
| title_sort | reinforcement learning based sequential control policy for multiple peg in hole assembly |
| topic | multiple peg-in-hole assembly deep reinforcement learning self-supervised contrastive learning |
| url | https://www.sciopen.com/article/10.26599/AIR.2024.9150043 |
| work_keys_str_mv | AT xinyuliu reinforcementlearningbasedsequentialcontrolpolicyformultiplepeginholeassembly AT chaozeng reinforcementlearningbasedsequentialcontrolpolicyformultiplepeginholeassembly AT chenguangyang reinforcementlearningbasedsequentialcontrolpolicyformultiplepeginholeassembly AT jianweizhang reinforcementlearningbasedsequentialcontrolpolicyformultiplepeginholeassembly |