Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models
Intelligent robot is the ultimate goal in the robotics field. Existing works leverage learning-based or optimization-based methods to accomplish human-defined tasks. However, the challenge of enabling robots to explore various environments autonomously remains unresolved. In this work, we propose a...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-05-01
|
| Series: | CAAI Artificial Intelligence Research |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/AIR.2024.9150037 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550184706736128 |
|---|---|
| author | Shoujie Li Ran Yu Tong Wu Junwen Zhong Xiao-Ping Zhang Wenbo Ding |
| author_facet | Shoujie Li Ran Yu Tong Wu Junwen Zhong Xiao-Ping Zhang Wenbo Ding |
| author_sort | Shoujie Li |
| collection | DOAJ |
| description | Intelligent robot is the ultimate goal in the robotics field. Existing works leverage learning-based or optimization-based methods to accomplish human-defined tasks. However, the challenge of enabling robots to explore various environments autonomously remains unresolved. In this work, we propose a framework named GExp, which endows robots with the capability of exploring and learning autonomously without human intervention. To achieve this goal, we devise modules including self-exploration, knowledge-base-building, and close-loop feedback based on foundation models. Inspired by the way that infants interact with the world, GExp encourages robots to understand and explore the environment with a series of self-generated tasks. During the process of exploration, the robot will acquire skills from experiences that are useful in the future. GExp provides robots with the ability to solve complex tasks through self-exploration. GExp work is independent of prior interactive knowledge and human intervention, allowing it to adapt directly to different scenarios, unlike previous studies that provided in-context examples as few-shot learning. In addition, we propose a workflow of deploying the real-world robot system with self-learned skills as an embodied assistant. Project website: GExp.com. |
| format | Article |
| id | doaj-art-58f5e9a8bfff4dd988680d28388d0be5 |
| institution | Kabale University |
| issn | 2097-194X 2097-3691 |
| language | English |
| publishDate | 2024-05-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | CAAI Artificial Intelligence Research |
| spelling | doaj-art-58f5e9a8bfff4dd988680d28388d0be52025-01-10T06:44:32ZengTsinghua University PressCAAI Artificial Intelligence Research2097-194X2097-36912024-05-013915003710.26599/AIR.2024.9150037Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation ModelsShoujie Li0Ran Yu1Tong Wu2Junwen Zhong3Xiao-Ping Zhang4Wenbo Ding5Shenzhen Ubiquitous Data Enabling Key Lab, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, ChinaShenzhen Ubiquitous Data Enabling Key Lab, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, ChinaShenzhen Ubiquitous Data Enabling Key Lab, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, ChinaDepartment of Physics and Chemistry, Faculty of Science and Technology, University of Macau, Macau 999078, ChinaShenzhen Ubiquitous Data Enabling Key Lab, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, ChinaShenzhen Ubiquitous Data Enabling Key Lab, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, ChinaIntelligent robot is the ultimate goal in the robotics field. Existing works leverage learning-based or optimization-based methods to accomplish human-defined tasks. However, the challenge of enabling robots to explore various environments autonomously remains unresolved. In this work, we propose a framework named GExp, which endows robots with the capability of exploring and learning autonomously without human intervention. To achieve this goal, we devise modules including self-exploration, knowledge-base-building, and close-loop feedback based on foundation models. Inspired by the way that infants interact with the world, GExp encourages robots to understand and explore the environment with a series of self-generated tasks. During the process of exploration, the robot will acquire skills from experiences that are useful in the future. GExp provides robots with the ability to solve complex tasks through self-exploration. GExp work is independent of prior interactive knowledge and human intervention, allowing it to adapt directly to different scenarios, unlike previous studies that provided in-context examples as few-shot learning. In addition, we propose a workflow of deploying the real-world robot system with self-learned skills as an embodied assistant. Project website: GExp.com.https://www.sciopen.com/article/10.26599/AIR.2024.9150037intelligent robotfoundation modelsself-exploring framework |
| spellingShingle | Shoujie Li Ran Yu Tong Wu Junwen Zhong Xiao-Ping Zhang Wenbo Ding Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models CAAI Artificial Intelligence Research intelligent robot foundation models self-exploring framework |
| title | Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models |
| title_full | Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models |
| title_fullStr | Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models |
| title_full_unstemmed | Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models |
| title_short | Growing from Exploration: A Self-Exploring Framework for Robots Based on Foundation Models |
| title_sort | growing from exploration a self exploring framework for robots based on foundation models |
| topic | intelligent robot foundation models self-exploring framework |
| url | https://www.sciopen.com/article/10.26599/AIR.2024.9150037 |
| work_keys_str_mv | AT shoujieli growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels AT ranyu growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels AT tongwu growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels AT junwenzhong growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels AT xiaopingzhang growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels AT wenboding growingfromexplorationaselfexploringframeworkforrobotsbasedonfoundationmodels |