SwinE-CA: BEV Perception Based on Dual Backbone of CNN and Transformer

Bird’s Eye View (BEV) can fuse the information between each different camera to accomplish a unified global scene representation. Many existing models use standard convolutional networks to extract image features. Convolutional operations are good at extracting local features, but still have limitations in capturing global feature representation. Recently, Transformer has been prominent in the vision community due to its excellent ability to capture global feature representations. In this paper, we propose a dual-backbone network model, SwinE-CA, which is based on the CNN and Transformer. This model is designed to extract multi-scale features from multi-view images and aggregate them into a shared Bird’s Eye View representation. Experiments on nuScenes dataset for BEV segmentation were conducted to verify the effectiveness of the proposed method. SwinE-CA achieved 39.4 and 36.9 mIoU with two different evaluation settings, an improvement of 1.2 and 0.5 mIoU, respectively. The combination of global and local features contributes to model learning.