This paper addresses the challenges in drone object detection—complex backgrounds and numerous small objects—aiming to improve detection accuracy without increasing parameter count. By analyzing YOLOv10n’s architecture, we propose SSA-YOLO, a lightweight enhanced version with two key improvements: the SSA (Small Shuffle Attention) module in the backbone strengthens small-object feature extraction and fusion via spatial attention, while the USKConv (UltraLight Selective Kernel Convolution) module in the Neck optimizes target localization and classification through dynamic feature processing. Experiments on Visdrone2019 show SSA-YOLO improves mAP50 by 2.2% over YOLOv10n, with only a 7.1% parameter increase, and maintains generalization across datasets, validating the proposed strategies. This work supports practical drone detection applications and offers insights for optimizing small-object models on mobile devices.