To address the issues of the existing frustum-based methods’ underutilization of image information in road 3D object detection as well as the lack of research on agricultural scenes, we constructed an object detection dataset using an 80-line LiDAR and a camera in a complex tractor road scene and proposed a new network called FrustumFusionNets (FFNets). Initially, we utilize the results of image-based 2D object detection to narrow down the search region in the 3D space of the point cloud. Next, we introduce a Gaussian mask to enhance the point cloud information. Then, we extract the features from the frustum point cloud and the crop image using the point cloud feature extraction pipeline and the image feature extraction pipeline, respectively. Finally, we concatenate and fuse the data features from both modalities to achieve 3D object detection. Experiments demonstrate that on the constructed test set of tractor road data, the FrustumFusionNetv2 achieves 82.28% and 95.68% accuracy in the 3D object detection of the two main road objects, cars and persons, respectively. This performance is 1.83% and 2.33% better than the original model. It offers a hybrid fusion-based multi-object, high-precision, real-time 3D object detection technique for unmanned agricultural machines in tractor road scenarios. On the KITTI validation set, the FrustumFusionNetv2 also demonstrates significant superiority in detecting road pedestrian objects compared with other frustum-based 3D object detection methods.