The real-time detection transformer has been widely used in infrared ship detection, but there are shortcomings in terms of computational complexity and feature fusion. Therefore, an infrared vessel detection model based on bidirectional selective path aggregation network was proposed in this paper. In this model, the StarNet was introduced to replace the traditional ResNet as the backbone network, and the lightweight of the model was achieved while feature extraction capabilities were maintained. It adopted an efficient additive attention mechanism to replace the original multi-head self-attention, reducing computational complexity and enhancing the perception accuracy of key ship areas. At the same time, to solve the problem of information loss in feature fusion at different scales, a bidirectional routing aggregation network optimization feature fusion strategy was proposed, which can better deal with ship targets of different sizes through semantic enhancement feature selection module and detailed perception feature fusion module. The experimental results show that the model exhibits excellent detection performance on the infrared ship dataset. It ensures detection accuracy and achieves higher real-time processing capabilities, and can effectively adapt to complex and changing sea surface monitoring environments.