6D Grasp with Point Cloud Fusion
Model-free 6D pose estimation and point-cloud completion for robotic grasping.
Because of irregularities on an object’s surface and changes in lighting, a single viewpoint often yields point clouds with gaps and errors at the edges. Those holes make grasp-pose estimation brittle. This project built a grasping pipeline that first estimates a 6D object pose, then uses that pose to complete the observed cloud before predicting a grasp.
A model-free pose network based on perspective matching estimates the pose of objects seen during training and a coarser pose for untrained objects. When only a sparse cloud is available, a refinement network tightens that estimate. The pose then drives ICP-based fusion: the model cloud fills missing regions in the observed cloud and filters noise along the silhouette.
Grasp poses are generated from the completed cloud with a grasping-direction network and a fast search strategy. The system was tested in simulation and on a 6-DoF robot with a RealSense camera and a ROS / Ubuntu 20.04 host.