GFENet: group-wise feature-enhanced network for steering angle prediction by fusing events and images

Existing end-to-end networks for steering angle prediction usually use images generated by standard cameras as input. However, standard cameras are susceptible to poor lighting conditions and motion blur, which is not conducive to training an accurate and robust end-to-end network. In contrast, biological vision-inspired event cameras overcome the aforementioned shortcomings with their unique working principle and offer significant advantages such as high temporal resolution, high dynamic range and low power consumption. Nevertheless, event cameras generate a lot of noise and are unable to provide texture information on static region. Therefore, these two types of cameras are complementary to each other to some extent. To explore the benefits of fusing information from these two types of cameras in autonomous driving tasks, we propose GFENet, an attention-based two-stream encoder-decoder architecture for steering angle prediction by combining events and images. Firstly, asynchronous and sparse events are converted into synchronous and dense event frames. Then, event frames and corresponding image frames are fed into two symmetric encoders to extract features. Next, We introduce a Group-Wise Feature-Enhanced (GEF) module that can refine features and suppress noise to guide the fusion of two modalities features at different levels. Finally, The final fused features are passed through a simple decoder to predict the steering angle. Experiments results on the DDD20 and EventScape datasets shows that our GFEFNet outperforms the state-of-the-art image-event fusion method.