The use of multi-sensor cooperative fusion technology is an important method to enhance traffic perception capabilities in complex environments. Most of the popular research is still focused on the fusion of LiDAR sensors and cameras. However, due to the inability of LiDAR sensors and cameras to directly obtain accurate speed supervision and poor environmental robustness. They cannot always maintain the accuracy of moving object perception and the perception ability under different restricted conditions. Given that radar not only takes into account the robustness in harsh weather, but also has reliable speed supervision. Therefore, we propose a new 3d object detection method that uses radar, LiDAR, and camera sensors to fuse in the Bird’s-Eye View (BEV). By encoding and filtering radar information, and accumulating point clouds in the time dimension, the expressive ability of sparse point clouds is improved. For the first time, the Grid View Transformer method is proposed, which realizes the mapping of radar sensor features to the BEV space. Finally, an adaptive hierarchical fusion method is designed to achieve effective cross-modal feature fusion for different modal features. Experimental results show that the method in this paper has obtained 71.0 mean average precision (mAP) and 73.6 nuScenes detection score (NDS) on the challenging nuScenes dataset. Particularly, the perception ability of the algorithm for object speed is particularly outstanding.