[Paper Notes] 6D Pose Estimation Papers from ECCV2020–2022

ECCV2020

01. CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation

Journal / Conference: ECCV2020 Author / Institution: Yann Labbé, École normale supérieure, CNRS, PSL Research University, Paris, France Keywords: Pose estimation Year: 2020 Code: https://github.com/ylabbe/cosypose

1 Objective

Use multi-view information to improve the accuracy and robustness of object pose estimation when the camera positions are unknown.

2 Method

(1) Generate initial 6D pose candidates for objects

Given a set of objects with known 3D models and a single image of the scene, we output a set of candidate detections for each object and, for each detection, the object’s 6D pose relative to the camera associated with the image.

(2) Match object candidates

Match objects visible in multiple views to obtain a single consistent scene.

(3) Refine the global scene

The 6D poses of all objects and cameras are optimized to minimize the global reprojection error.

3 Thoughts

(1) Innovations

• Proposes a render-and-compare-based method for single-view, single-object 6D pose estimation to generate object pose hypotheses in each image.
• Develops a robust RANSAC-based method to match individual object pose hypotheses across different images and uses these object-level correspondences to recover the relative positions between cameras.
• Develops a global optimization method based on object-level bundle adjustment to minimize the reprojection error across all views and improve noisy single-view object poses.

(2) Practicality

Inferring objects’ 6D poses from multiple views has limited practicality for grasping scenarios.

ECCV2022

01. DProST: Dynamic Projective Spatial Transformer Network for 6D Pose Estimation

Journal / Conference: ECCV2022 Author / Institution: Keywords: Year: 2022 Code: https://github.com/parkjaewoo0611/DProST Dataset: LINEMOD

1 Objective

Proposes a new pose estimation system based on a projective grid.

2 Method

• Uses a deep neural network to extract features from an RGB image and predict the object’s position and size.
• Generates a conical ray grid in camera space according to the predicted position and size, then inversely transforms it into object space.
• Uses a reference image and mask to extract texture features from the object model or reconstructed features, then maps them onto the transformed grid.
• Uses bilinear interpolation to sample the mapped texture features, obtaining a reconstructed image that is compared with the input image.
• Uses loss functions based on grid distance and grid matching to optimize the network parameters and pose parameters.

3 Thoughts

A deep neural network extracts features, a projective grid reconstructs the image, and loss functions optimize the parameters.

02. DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation

Journal / Conference: ECCV2022 Author / Institution: Hongyang Li, South China University of Technology, Guangzhou, China Keywords: Pose estimation Year: 2022 Code: https://github.com/Gorilla-Lab-SCUT/DCL-Net Dataset: LINEMOD

1 Objective

Directly estimate 6D object poses from point correspondences rather than using indirect correspondence learning objectives.

2 Method

The main method in this paper is a new deep correspondence learning network (DCL-Net). It uses dual feature disentangling and alignment (FDA) modules to establish partial-to-partial and complete-to-complete correspondences between the camera and object coordinate systems in feature space. The specific steps are as follows:

• First, point feature maps are extracted separately for a partial object observation and its CAD model.
• Next, two FDA modules are designed to establish partial-to-partial and complete-to-complete correspondences, respectively. Specifically, each FDA module takes two point feature maps as input and disentangles each feature map into an independent pose feature map and matching feature map. It then uses the matching feature maps to learn an attention map for establishing deep correspondences. Finally, the pose feature maps and matching feature maps from the two systems are aligned and paired according to the attention map, producing pose feature pairs and matching feature pairs.
• The two sets of correspondences produced by the two FDA modules are then fused because they have complementary advantages. The fused matching feature pairs are used to learn confidence scores that measure the quality of the deep correspondences. At the same time, the confidence scores are used to weight the fused pose feature pairs and directly regress the object pose.
• Finally, a confidence-based pose refinement network is proposed to further improve pose accuracy iteratively.

3 Thoughts

A point-feature-based method.

03. Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation

Journal / Conference: ECCV2022 Author / Institution: Yinlin Hu, EPFL CVLab, Lausanne, Switzerland Keywords: Pose estimation; limited-data scenarios Year: 2022 Code: https://github.com/cvlab-epfl/perspective-flow-aggregation Dataset: LINEMOD

1 Objective

Train a 6D object pose estimation model using synthetic images or a small number of real images when data are limited.

2 Method

• First, synthetic images and real images (if available) are used to train a deep-learning-based feature extractor that extracts features related to the object pose from the input image.
• Next, synthetic images and real images (if available) are used to train a perspective-flow-based module that maps feature points in the input image onto the target object model. Perspective flow refers to the displacement of feature points across different viewpoints caused by relative motion between the camera and the object.
• Finally, a technique called perspective flow aggregation fuses multiple perspective flows, and the optimal 6D object pose is solved using least squares.

3 Thoughts

A projection- and perspective-based method.

04. Learning-Based Point Cloud Registration for 6D Object Pose Estimation in the Real World

Journal / Conference: ECCV2022 Author / Institution: Zheng Dang, CVLab, EPFL, Lausanne, Switzerland Keywords: Pose estimation Year: 2022 Code: https://github.com/AnsonYanxin/MatchNorm Dataset: LINEMOD

1 Objective

2 Method

• First, it proposes a deep-learning-based point cloud matching module that extracts features from the source and target point clouds and computes a similarity matrix between the two point clouds.
• Next, it proposes a normalization-based point cloud alignment module that uses the similarity matrix to find the optimal rigid transformation matrix, minimizing the distance between the source and target point clouds.

3 Thoughts

The code is incomplete.