In response to the current challenges of large parameter size and low detection accuracy in SAR ship detection models, this paper proposes an improved YOLOv8 model. The model integrates the Swin-Transformer architecture into YOLOv8 through an adaptive feature fusion method, enhancing the model’s global information perception and detection accuracy. Additionally, it employs a top-down unidirectional semantic pyramid and a lightweight detection head structure to achieve model lightweighting. Experimental comparisons on the HRSID dataset show that, compared to the benchmark algorithm, the proposed algorithm increases mAP50 by 1.2% and reduces model parameter size by 75%. Compared to current mainstream algorithms, this algorithm offers better detection performance and a lower model parameter size, effectively meeting the requirements for model lightweighting and high detection accuracy in SAR image detection terminal devices.