An 18-DOF hand integrating force–position multimodal perception using a monocular camera
Abstract The anthropomorphic hand plays a crucial role in human-machine interaction tasks. However, there are very few hands that realize multimodal perception with high degrees of freedom (DOF) in a low-cost way. Here, we present a dexterous hand that achieves multimodal sensing solely through a ca...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-62122-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849764084621246464 |
|---|---|
| author | Shiwei Chen Jiapeng Li Zhiming Deng Peiji Wang Cheng Wei Xibin Cao |
| author_facet | Shiwei Chen Jiapeng Li Zhiming Deng Peiji Wang Cheng Wei Xibin Cao |
| author_sort | Shiwei Chen |
| collection | DOAJ |
| description | Abstract The anthropomorphic hand plays a crucial role in human-machine interaction tasks. However, there are very few hands that realize multimodal perception with high degrees of freedom (DOF) in a low-cost way. Here, we present a dexterous hand that achieves multimodal sensing solely through a camera. The hand has 18 DOF but does not require any position or force sensors, making it cost-effective and easy to manufacture. We develop an integrated forearm for the hand that provides both actuation and multimodal sensing information simultaneously. This includes the 18 joint angles, 5 fingertip positions and contact forces, and information on object softness and contour. The core principle of perception is that the camera can track the displacement and tension of all tendons simultaneously. The multimodal perception model is developed by characterizing tendon properties and coupling them with the hand dynamics. Experiments indicate that our hand has potential in multimodal sensing and dexterity. |
| format | Article |
| id | doaj-art-e3fd642fcf104d8aaa29684cc0353abb |
| institution | DOAJ |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-e3fd642fcf104d8aaa29684cc0353abb2025-08-20T03:05:14ZengNature PortfolioNature Communications2041-17232025-07-0116111410.1038/s41467-025-62122-0An 18-DOF hand integrating force–position multimodal perception using a monocular cameraShiwei Chen0Jiapeng Li1Zhiming Deng2Peiji Wang3Cheng Wei4Xibin Cao5School of Aerospace, Harbin Institute of TechnologySchool of Aerospace, Harbin Institute of TechnologySchool of Aerospace, Harbin Institute of TechnologySchool of Aerospace, Harbin Institute of TechnologySchool of Aerospace, Harbin Institute of TechnologySchool of Aerospace, Harbin Institute of TechnologyAbstract The anthropomorphic hand plays a crucial role in human-machine interaction tasks. However, there are very few hands that realize multimodal perception with high degrees of freedom (DOF) in a low-cost way. Here, we present a dexterous hand that achieves multimodal sensing solely through a camera. The hand has 18 DOF but does not require any position or force sensors, making it cost-effective and easy to manufacture. We develop an integrated forearm for the hand that provides both actuation and multimodal sensing information simultaneously. This includes the 18 joint angles, 5 fingertip positions and contact forces, and information on object softness and contour. The core principle of perception is that the camera can track the displacement and tension of all tendons simultaneously. The multimodal perception model is developed by characterizing tendon properties and coupling them with the hand dynamics. Experiments indicate that our hand has potential in multimodal sensing and dexterity.https://doi.org/10.1038/s41467-025-62122-0 |
| spellingShingle | Shiwei Chen Jiapeng Li Zhiming Deng Peiji Wang Cheng Wei Xibin Cao An 18-DOF hand integrating force–position multimodal perception using a monocular camera Nature Communications |
| title | An 18-DOF hand integrating force–position multimodal perception using a monocular camera |
| title_full | An 18-DOF hand integrating force–position multimodal perception using a monocular camera |
| title_fullStr | An 18-DOF hand integrating force–position multimodal perception using a monocular camera |
| title_full_unstemmed | An 18-DOF hand integrating force–position multimodal perception using a monocular camera |
| title_short | An 18-DOF hand integrating force–position multimodal perception using a monocular camera |
| title_sort | 18 dof hand integrating force position multimodal perception using a monocular camera |
| url | https://doi.org/10.1038/s41467-025-62122-0 |
| work_keys_str_mv | AT shiweichen an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT jiapengli an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT zhimingdeng an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT peijiwang an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT chengwei an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT xibincao an18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT shiweichen 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT jiapengli 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT zhimingdeng 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT peijiwang 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT chengwei 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera AT xibincao 18dofhandintegratingforcepositionmultimodalperceptionusingamonocularcamera |