LPL-VIO: monocular visual-inertial odometry with deep learning-based point and line features

Visual-inertial SLAM (VINS) has garnered substantial interest in the field of mobile robotics due to its robustness and cost-effectiveness. The incorporation of both point and line features in visual SLAM has shown significant performance improvements compared to relying solely on point features. Furthermore, learning-based features consistently outperform traditional hand-crafted features across a diverse range of tasks. Leveraging all these observations, we propose a novel deep learning technique, termed learning-based point and line visual-inertial odometry (LPL-VIO), which is aimed to enhance robustness and performance of traditional systems. LPL-VIO takes a 2-stage approach, including a deep learning front-end and a traditional nonlinear optimization back-end. In the first stage (front-end), we employ proposed networks for points and lines to extract and track deep learning-based point and line features in visual images, while implementing IMU preintegration. In the second stage (back-end), we adopt a sliding window strategy to tightly couple point and line re-projection errors with IMU preintegration residuals. These residuals are optimized using bundle adjustment for precise global pose estimation. Extensive experiments are conducted on both public EuRoC datasets and our own real-world dataset collected in challenging outdoor environments. Qualitative and quantitative results show that our method exhibits strong competitiveness compared with the existing point–line techniques, and outperforms the popular VINS-Mono.