<p>Unmanned Aerial Vehicle (UAV) image object detection is a challenging research field with high research significance in practical applications. However, the relatively high imaging height of UAVs results in a high proportion and dense distribution of small objects in their aerial images. The method of transmitting large volumes of data from UAVs to a central server for centralized training has a high communication overhead and leakage of data privacy. To address these issues, we propose FedUD, a novel small object detection framework based on Federated Learning (FL), to assist UAV clusters for collaborative object detection. First, we propose a local model UODNet based on YOLOv8 for small object detection in the FedUD framework. UODNet achieves better detection accuracy by adding a tiny object prediction head and three designed modules for small object features. Second, FedUD contains a new FL algorithm in which the server freezes the model layers with low similarity and locally utilizes the historical knowledge for parameterized regularization. The algorithm can solve the problem of accuracy reduction caused by the traditional FL after the aggregation of models with complex structures. Extensive experiments demonstrate the effectiveness and advancement of our proposed method. Specifically, UODNet-s can achieve 49.9% mean average precision (mAP50) on the VisDrone-DET2019 dataset and 38.4% mAP50 on the UAVDT dataset, which is improved by 9.7% and 4.1%, respectively, compared to the baseline method. FedUD improves mAP50 by 4.7%−5.2% on the VisDrone-DET2019 dataset compared with the traditional FL baseline methods, which is more robust during complex model aggregation.</p>