The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment
The teleoperation of robot arms can prevent users from working in hazardous environments, but current teleoperation uses a 2D display and controls the end effector of robot arms, which introduces the problem of a limited view and complex operations. In this study, a teleoperation method for robot ar...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/7/3549 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850188208008069120 |
|---|---|
| author | Yan Wu Bin Zhao Qi Li |
| author_facet | Yan Wu Bin Zhao Qi Li |
| author_sort | Yan Wu |
| collection | DOAJ |
| description | The teleoperation of robot arms can prevent users from working in hazardous environments, but current teleoperation uses a 2D display and controls the end effector of robot arms, which introduces the problem of a limited view and complex operations. In this study, a teleoperation method for robot arms is proposed, which can control the robot arm by interacting with the digital twins of objects. Based on the objects in the workspace, this method generates a virtual scene containing digital twins. Users can observe the virtual scene from any direction and move the digital twins of the objects at will to control the robot arm. This study compared the proposed method and the traditional method, which uses a 2D display and a game controller, through a pick-and-place task. The proposed method achieved 45% lower scores in NASA-TLX and 31% higher scores in SUS than traditional teleoperation methods. The results indicate that the proposed method can reduce the workload and improve the usability of teleoperation. |
| format | Article |
| id | doaj-art-be8a1f7c7f4847bd894a29a598488768 |
| institution | OA Journals |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-be8a1f7c7f4847bd894a29a5984887682025-08-20T02:15:55ZengMDPI AGApplied Sciences2076-34172025-03-01157354910.3390/app15073549The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality EnvironmentYan Wu0Bin Zhao1Qi Li2School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, ChinaSchool of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, ChinaSchool of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, ChinaThe teleoperation of robot arms can prevent users from working in hazardous environments, but current teleoperation uses a 2D display and controls the end effector of robot arms, which introduces the problem of a limited view and complex operations. In this study, a teleoperation method for robot arms is proposed, which can control the robot arm by interacting with the digital twins of objects. Based on the objects in the workspace, this method generates a virtual scene containing digital twins. Users can observe the virtual scene from any direction and move the digital twins of the objects at will to control the robot arm. This study compared the proposed method and the traditional method, which uses a 2D display and a game controller, through a pick-and-place task. The proposed method achieved 45% lower scores in NASA-TLX and 31% higher scores in SUS than traditional teleoperation methods. The results indicate that the proposed method can reduce the workload and improve the usability of teleoperation.https://www.mdpi.com/2076-3417/15/7/3549human–robot interactionmixed realityteleoperationrobotic manipulation |
| spellingShingle | Yan Wu Bin Zhao Qi Li The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment Applied Sciences human–robot interaction mixed reality teleoperation robotic manipulation |
| title | The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment |
| title_full | The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment |
| title_fullStr | The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment |
| title_full_unstemmed | The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment |
| title_short | The Teleoperation of Robot Arms by Interacting with an Object’s Digital Twin in a Mixed Reality Environment |
| title_sort | teleoperation of robot arms by interacting with an object s digital twin in a mixed reality environment |
| topic | human–robot interaction mixed reality teleoperation robotic manipulation |
| url | https://www.mdpi.com/2076-3417/15/7/3549 |
| work_keys_str_mv | AT yanwu theteleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment AT binzhao theteleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment AT qili theteleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment AT yanwu teleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment AT binzhao teleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment AT qili teleoperationofrobotarmsbyinteractingwithanobjectsdigitaltwininamixedrealityenvironment |