Project Website: https://graspgen.github.io/
Code: https://github.com/NVlabs/GraspGen/
Abstract: Grasping is a fundamental robot skill, yet despite significant research advancements, learning-based 6-DOF grasping approaches are still not turnkey and struggle to generalize across different embodiments and in-the-wild settings. We build upon the recent success on modeling the object-centric grasp generation process as an iterative diffusion process. Our proposed framework - GraspGen - consists of a Diffusion-Transformer architecture that enhances grasp generation, paired with an efficient discriminator to score and filter sampled grasps. We introduce a novel and performant on-generator training recipe for the discriminator. To scale GraspGen to both objects and grippers, we release a new simulated dataset consisting of over 53 million grasps. We demonstrate that GraspGen outperforms prior methods in simulations with singulated objects across different grippers, achieves state-of-the-art performance on the FetchBench grasping benchmark, and performs well on a real robot with noisy visual observations.
Model Architecture:
Architecture Type: Diffusion Model, Point Cloud network. See paper for more details.
Input:
Input Type(s): Object partial point cloud X, Number of grasps to sample (B)
Input Format(s): Point Cloud (N X 3) where N is the number of points
Input Parameters: 3D
Other Properties Related to Input: Point cloud needs to be in the form (N X xyz) where N=2048 is the number of points.
Output:
Output Type(s): Grasp Poses; Corresponding confidence scores
Output Format: Homogenous Transformation matrices; score is a scalar value from 0 to 1
Output Parameters: [B, 4, 4] where B is the number of generated grasp poses; [B, 1] confidence score
Other Properties Related to Output: