End-to-End LiDAR-Camera Self-Calibration Using Vision Transformer
摘要
This paper presents a novel end-to-end deep learning framework for extrinsic calibration between LiDAR and camera sensors. The core contribution is the integration of a Transformer-based global attention architecture and a cross-attention mechanism, enabling effective learning of semantic and spatial correspondences between RGB images and 3D point clouds. The Transformer architecture effectively models long-range dependencies, whereas the cross-attention module reinforces intermodal interactions. This combination enables the network to capture global context and semantic alignment across sensors, which conventional CNN-based methods often fail to achieve. Consequently, the proposed method maintains robust and accurate calibration performance even under challenging conditions, such as occlusions, perspective mismatches, and sparse or noisy inputs. Evaluated on the widely used KITTI odometry dataset, the model achieves mean translation and rotation errors of 1.37 cm and 0.09°, respectively, demonstrating a marked improvement in extrinsic calibration precision. Furthermore, ablation studies confirm that both the Transformer and cross-attention modules are critical to performance gains, demonstrating their effectiveness in enhancing multimodal feature matching and calibration accuracy.