Task-Oriented Dexterous Grasping via Grasp Taxonomy

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026
Hyunwoo Kim1, Jeonghwan Choi1, Dohyun Kim2, Juntae Hwang3, Ilsung Jin4, Donghan Kim1,*
1Dept. of Electronic Engineering (AgeTech-Service Convergence Major), Kyung Hee University
2Dept. of Electronic Engineering, Kyung Hee University
3Dept. of Convergence Security, Kyung Hee University
4Dept. of Artificial Intelligence, Kyung Hee University
*Corresponding author
Paper arXiv (soon) Code (soon) Video

We use a grasp taxonomy as an intermediate representation between task intention and hand–object contact, converting dexterous grasps into a consistent two-point opposition (OP1, OP2) to extend task-oriented parallel-gripper pipelines to dexterous hands.

Abstract

Task-oriented grasping requires selecting grasp strategies that match the intended use of an object, but most existing approaches are limited to parallel grippers and do not fully exploit the contact diversity of dexterous hands. In this work, we propose a task-oriented dexterous grasp generation framework that bridges the gap between task intention and executable multi-finger grasps through a grasp taxonomy and contact representation. We first convert the dexterous hand grasp taxonomy into a consistent 2-point opposition representation using the Contact Web, enabling alignment with task-oriented parallel gripper grasp poses. We then predict a taxonomy-conditioned Contact Semantic Map from the object point cloud, grasp pose, and grasp taxonomy, and use it to guide transformer-based dexterous grasp generation. In real-world experiments on unseen household objects, our approach successfully translates diverse grasp taxonomies into stable and task-consistent dexterous executions. These results demonstrate that a grasp taxonomy is an effective intermediate representation for extending existing task-oriented grasping pipelines to dexterous hands.

Overview of the proposed pipeline

Overview of the proposed pipeline.

Given an instruction and an object point cloud, a task-oriented grasping (TOG) model predicts task-relevant parallel gripper grasp poses, while an LLM performs grasp type reasoning to select an appropriate grasp taxonomy. The semantic contact prediction module estimates a Contact Semantic Map conditioned on the grasp pose and the selected taxonomy. Finally, the grasp generation module synthesizes executable dexterous grasps from the predicted contact distributions.

Conversion from dexterous grasp taxonomy to a parallel gripper representation

Taxonomy-to-opposition conversion.

Left: Representative grasp types from the Feix taxonomy grouped into Power, Intermediate, and Precision categories. Middle: Contact Web–based conversion that reduces multi-finger contacts into a two-point opposition structure (OP1, OP2). For typical grasps, OP1 is selected from the VF1 group and OP2 is determined from the contact web along the contact vector vc. For ring-like grasps, where palm contacts form a wrap configuration, OP2 is defined as the centroid of finger contact points and OP1 is selected from the opposing palm region. Right: Example dexterous grasp configurations aligned with the converted parallel gripper pose for each taxonomy.

Contact Semantic-based dexterous grasp generation framework

Contact Semantic-based dexterous grasp generation framework.

The pipeline consists of two stages: (top) FiLM-conditioned Contact Semantic Map Prediction, which estimates contact distributions based on the object point cloud, grasp pose, and grasp taxonomy; and (bottom) Grasp Generation, which utilizes a transformer-based module for dexterous grasp synthesis followed by test-time optimization.

Results

Successful task-oriented dexterous grasp examples

Successful task-oriented dexterous grasp examples across different grasp taxonomies.

Real-world experiments are performed on five unseen household objects (glue gun, mug, pitcher, knife, and spatula) using a UR3 manipulator, an Allegro Hand V5, and an Intel RealSense D435i depth camera. Our approach successfully translates diverse grasp taxonomies into stable and task-consistent dexterous executions.

BibTeX

@inproceedings{kim2026taskoriented,
  title     = {Task-Oriented Dexterous Grasping via Grasp Taxonomy},
  author    = {Kim, Hyunwoo and Choi, Jeonghwan and Kim, Dohyun and
               Hwang, Juntae and Jin, Ilsung and Kim, Donghan},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots
               and Systems (IROS)},
  year      = {2026}
}

Acknowledgment

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the Convergence Security Core Talent Training Business Support Program (IITP-2023-RS-2023-00266615) supervised by the IITP; the BK21 plus program "AgeTech-Service Convergence Major" through the National Research Foundation (NRF) funded by the Ministry of Education of Korea; the IITP grant funded by the Korea government (MSIT) (No. RS-2022-00155911); the Technology Innovation Program funded by MOTIE, Korea; and the Smart Farm Innovation Technology Development Program through IPET and KosFarm, funded by MAFRA, MSIT, and RDA.