Lightweight Multimodal Fusion for Autonomous Navigation via Deep Reinforcement Learning

We present a novel lightweight multimodal fusion mobile robot navigation system, which integrates RGB cameras and 2D LiDARs to overcome the limitations of traditional 2D LiDARs in detecting obstacles confined to a single plane. The RGB camera captures images, and image features are extracted using an autoencoder. To enhance the focus on obstacle features during the encoding process, we introduce a spatial attention mechanism and utilize threshold processing to derive obstacle masks from the RGB images. The obstacle mask loss function is then incorporated into the loss function to amplify attention on obstacle features. By combining the extracted image features with processed 2D LiDAR data, we effectively train the mobile robot for navigation and obstacle avoidance in a simulated environment. The experimental results demonstrate that our proposed multimodal fusion mobile robot navigation system has a success rate of up to 99%, and the addition of spatial attention and obstacle mask loss function improves the mobile robot’s navigation ability.