<p>Deep learning-based object detection algorithms have recently found extensive applications in various fields, including autonomous driving, medical diagnosis, and face recognition. Although many algorithms have achieved high accuracy in detecting large- and medium-sized objects, the accuracy of images from the aerial view of UAVs (unmanned aerial vehicles) is unsatisfactory. It decreases significantly due to its low feature information and complex background. In this paper, we propose a novel algorithm called CSF-YOLO (cross-scale feature fusion YOLO), which is specifically designed for small object detection. The CSF-YOLO algorithm builds upon the YOLOv8 single-stage detection framework, incorporating several enhancements. Firstly, a backbone is proposed to enhance the network’s extraction of shallow information. It uses full-dimensional dynamic convolution for encoding, effectively improving the network’s ability to extract features from small objects. Secondly, a feature fusion module is proposed to enhance the network’s multilayer feature fusion capabilities further. Thirdly, a neck structure is proposed to increase the extraction of shallow information and improve the network’s ability to mine the location information of small objects. In addition, attention module methods are used to improve the performance of the network. Finally, experiments are conducted on the VisDrone2021 dataset, which involves many small objects captured by UAVs. Compared to the baseline model (YOLOv8s), mAP (the mean average precision) of CSF-YOLO increases from 23.8 to 28.3%, and FPS (frames per second) is 62. These comparison experiments further confirm the superior performance of our proposed method in enhancing the accuracy of small object detection.</p>