<p>This paper presents a novel end-to-end deep learning framework for extrinsic calibration between LiDAR and camera sensors. The core contribution is the integration of a Transformer-based global attention architecture and a cross-attention mechanism, enabling effective learning of semantic and spatial correspondences between RGB images and 3D point clouds. The Transformer architecture effectively models long-range dependencies, whereas the cross-attention module reinforces intermodal interactions. This combination enables the network to capture global context and semantic alignment across sensors, which conventional CNN-based methods often fail to achieve. Consequently, the proposed method maintains robust and accurate calibration performance even under challenging conditions, such as occlusions, perspective mismatches, and sparse or noisy inputs. Evaluated on the widely used KITTI odometry dataset, the model achieves mean translation and rotation errors of 1.37&#xa0;cm and 0.09°, respectively, demonstrating a marked improvement in extrinsic calibration precision. Furthermore, ablation studies confirm that both the Transformer and cross-attention modules are critical to performance gains, demonstrating their effectiveness in enhancing multimodal feature matching and calibration accuracy.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

End-to-End LiDAR-Camera Self-Calibration Using Vision Transformer

  • Jun-yeong Kim,
  • Jae-ho Seong,
  • Tae-hyeong Kim,
  • Bong-seob Kim,
  • Kyung-su Yun,
  • Seok-cheol Kee

摘要

This paper presents a novel end-to-end deep learning framework for extrinsic calibration between LiDAR and camera sensors. The core contribution is the integration of a Transformer-based global attention architecture and a cross-attention mechanism, enabling effective learning of semantic and spatial correspondences between RGB images and 3D point clouds. The Transformer architecture effectively models long-range dependencies, whereas the cross-attention module reinforces intermodal interactions. This combination enables the network to capture global context and semantic alignment across sensors, which conventional CNN-based methods often fail to achieve. Consequently, the proposed method maintains robust and accurate calibration performance even under challenging conditions, such as occlusions, perspective mismatches, and sparse or noisy inputs. Evaluated on the widely used KITTI odometry dataset, the model achieves mean translation and rotation errors of 1.37 cm and 0.09°, respectively, demonstrating a marked improvement in extrinsic calibration precision. Furthermore, ablation studies confirm that both the Transformer and cross-attention modules are critical to performance gains, demonstrating their effectiveness in enhancing multimodal feature matching and calibration accuracy.