EA-Repose: Efficient and Accurate Feature-Metric-Based 6D Object Pose Refinement via Deep Reinforcement Learning

Recently, many works focus on 6D object pose refinement with a single RGB image. Most of them apply the differentiable Levenberg-Marquardt (LM) algorithm as the solver. However, they may easily ignore the importance of the damping parameter \(\lambda \) factor, which affects the accuracy and efficiency of prediction. In this paper, we present a coarse-to-fine feature-metric-based 6D object pose refinement framework, which utilizes the intermediate layers to predict \(\lambda \) combined with Region of Interest(ROI) alignment. To improve the speed without losing accuracy, we propose adaptive cropping during the prediction stage. Moreover, current works have not analyzed the control process during the whole iteration process. We propose to use deep reinforcement learning to fit this non-differentiable process, which can reduce redundant steps during the prediction stage. Finally, with a simple VGG-16 as the backbone, our algorithm with no iteration control learning achieves close performance with RNNPose [12] (state-of-the-art object pose refinement method) on Linemod,Linemod Occlusion and YCB-Video datasets, while the speed is 5 \(\times \) faster than it. Moreover, our full algorithm with TensorRT acceleration runs in about 80 FPS, the speed of which is close to RePose [11], but the accuracy is better than it especially for initial poses with large errors. The code is available at https://github.com/NiPeiyuan/EARePOSE.git . The supplementary material is available at here ( https://github.com/NiPeiyuan/EARePOSE/blob/main/supplimentary.pdf ).