Object-to-Manipulation Graph for Affordance Navigation
Object navigation, whose goal is to let the agent to reach some places (or objects), has been a popular topic in embodied Artificial Intelligence (AI) researches. However, in our real-world applications, it is more practical to find the targets with particular goals, raising the new requirements of...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-05-01
|
| Series: | CAAI Artificial Intelligence Research |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/AIR.2024.9150032 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841550188369412096 |
|---|---|
| author | Xinhang Song Bohan Wang Liye Dong Gongwei Chen Xinyun Hu Shuqiang Jiang |
| author_facet | Xinhang Song Bohan Wang Liye Dong Gongwei Chen Xinyun Hu Shuqiang Jiang |
| author_sort | Xinhang Song |
| collection | DOAJ |
| description | Object navigation, whose goal is to let the agent to reach some places (or objects), has been a popular topic in embodied Artificial Intelligence (AI) researches. However, in our real-world applications, it is more practical to find the targets with particular goals, raising the new requirements of finding the places to achieve the particular functions. In this paper, we define a new task of affordance navigation, whose goal is to find possible places to accomplish the required functions, achieving some particular effects. We first introduce a new dataset for affordance navigation, collected by the proposed affordance algorithm. In order to avoid the high cost of labor, the groundtruth of each episode which is annotated with the interaction data provided by the AI2-THOR simulator. In addition, we also propose an affordance navigation framework, where an Object-to-Manipulation Graph (OMG) is constructed and optimized to emphasize the corresponding nodes (including object nodes and manipulation nodes). Finally, a navigation policy is implemented (trained by reinforcement learning) to guide the navigation to the target places. Experimental results on AI2-THOR simulator illustrate the effectiveness of the proposed approach, which achieves significant gains of 14.0% and 11.7% (on success rate and Success weighted by Path Length (SPL), respectively) over the baseline model. |
| format | Article |
| id | doaj-art-67377843db3e4d1aba4566904cbb0cf8 |
| institution | Kabale University |
| issn | 2097-194X 2097-3691 |
| language | English |
| publishDate | 2024-05-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | CAAI Artificial Intelligence Research |
| spelling | doaj-art-67377843db3e4d1aba4566904cbb0cf82025-01-10T06:44:32ZengTsinghua University PressCAAI Artificial Intelligence Research2097-194X2097-36912024-05-013915003210.26599/AIR.2024.9150032Object-to-Manipulation Graph for Affordance NavigationXinhang Song0Bohan Wang1Liye Dong2Gongwei Chen3Xinyun Hu4Shuqiang Jiang5Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaKey Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, ChinaObject navigation, whose goal is to let the agent to reach some places (or objects), has been a popular topic in embodied Artificial Intelligence (AI) researches. However, in our real-world applications, it is more practical to find the targets with particular goals, raising the new requirements of finding the places to achieve the particular functions. In this paper, we define a new task of affordance navigation, whose goal is to find possible places to accomplish the required functions, achieving some particular effects. We first introduce a new dataset for affordance navigation, collected by the proposed affordance algorithm. In order to avoid the high cost of labor, the groundtruth of each episode which is annotated with the interaction data provided by the AI2-THOR simulator. In addition, we also propose an affordance navigation framework, where an Object-to-Manipulation Graph (OMG) is constructed and optimized to emphasize the corresponding nodes (including object nodes and manipulation nodes). Finally, a navigation policy is implemented (trained by reinforcement learning) to guide the navigation to the target places. Experimental results on AI2-THOR simulator illustrate the effectiveness of the proposed approach, which achieves significant gains of 14.0% and 11.7% (on success rate and Success weighted by Path Length (SPL), respectively) over the baseline model.https://www.sciopen.com/article/10.26599/AIR.2024.9150032navigationaffordancemanipulationgraph neural network |
| spellingShingle | Xinhang Song Bohan Wang Liye Dong Gongwei Chen Xinyun Hu Shuqiang Jiang Object-to-Manipulation Graph for Affordance Navigation CAAI Artificial Intelligence Research navigation affordance manipulation graph neural network |
| title | Object-to-Manipulation Graph for Affordance Navigation |
| title_full | Object-to-Manipulation Graph for Affordance Navigation |
| title_fullStr | Object-to-Manipulation Graph for Affordance Navigation |
| title_full_unstemmed | Object-to-Manipulation Graph for Affordance Navigation |
| title_short | Object-to-Manipulation Graph for Affordance Navigation |
| title_sort | object to manipulation graph for affordance navigation |
| topic | navigation affordance manipulation graph neural network |
| url | https://www.sciopen.com/article/10.26599/AIR.2024.9150032 |
| work_keys_str_mv | AT xinhangsong objecttomanipulationgraphforaffordancenavigation AT bohanwang objecttomanipulationgraphforaffordancenavigation AT liyedong objecttomanipulationgraphforaffordancenavigation AT gongweichen objecttomanipulationgraphforaffordancenavigation AT xinyunhu objecttomanipulationgraphforaffordancenavigation AT shuqiangjiang objecttomanipulationgraphforaffordancenavigation |