Contents
  1. 01. 6D Pose Estimation for Bin-Picking based on Improved Mask R-CNN and DenseFusion
  2. 1 Problem
  3. 2 Method
  4. 3 Thoughts
  5. 02. A Lightweight Two-End Feature Fusion Network for Object 6D Pose Estimation
  6. 1 Problem
  7. 2 Method
  8. 3 Thoughts

01. 6D Pose Estimation for Bin-Picking based on Improved Mask R-CNN and DenseFusion

Journal / Conference: 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) Author / Institution: Hesheng Wang, Shanghai Jiao Tong University Keywords: pose estimation, instance segmentation, MaskRCNN, DenseFusion Year: 2021 Code: None

1 Problem

Apply instance segmentation and pose estimation algorithms to industrial robotic grasping.

2 Method

Use a two-stage neural network to regress the 6D pose from an input RGB-D image.

Method

(1) Dataset Generation

Since learning-based algorithms require large labeled datasets, this paper presents a process for generating a virtual dataset of industrial parts. It uses Blender for physics simulation, adjusts the rendering parameters, and generates diverse, high-quality RGB images along with segmentation masks and 6D pose labels.

(2) Instance Segmentation

ResNeXt and MaskRCNN are used for object detection and instance segmentation.

(3) Pose Estimation

DenseFusion is used to predict the 6D pose. To improve performance, a NonLocal module—the green block in the figure—is added, allowing the network to learn spatial structure from the extracted features and effectively establish connections between point features. (It doesn’t seem to do much.)

Method (2)

3 Thoughts

A virtual dataset could help improve the robustness of the training results and is worth trying.

02. A Lightweight Two-End Feature Fusion Network for Object 6D Pose Estimation

Journal / Conference: Machines Author / Institution: Ligang Zuo, University of Science and Technology Beijing Keywords: pose estimation, feature fusion Year: 2022 Code: No code

1 Problem

Propose a lightweight pose estimation model for deployment on mobile devices.

2 Method

  1. Use the PointnoProblemNet network to extract point cloud features
  2. Fuse the point cloud features with the image features at the pixel level
  3. Use a CNN for feature extraction
  4. Estimate the pose for each feature and select the one with the highest confidence as the final result (Isn’t this just the DenseFusion approach, except that PointNet is replaced with PointnoProblem?)

Use depthwise separable convolutions instead of standard convolutions; that is, replace the ResNet feature extraction component with MobileNetv2 to reduce the number of model parameters.

3 Thoughts

It may be worth trying MobileNetv2 as the feature extraction network, since the device in use is also mobile.