Infrared aerial images often have low contrast, lack texture details and possess a small target scale. All above these bring great challenges to infrared target detection. In this paper, we propose a two-stage target detection and recognition algorithm inspired by eagle eye. Firstly, salient target regions are extracted by using the visual attention mechanism of eagle eye, and infrared target saliency is utilized to guide the target detection algorithm for further detection. Secondly, in view of the poor quality of infrared imaging, we propose to utilize the lateral inhibition mechanism to improve the image quality. Finally, considering the drastic change of target scale in aerial images, we progressively fuse large, medium and small scale features in the feature fusion network. Experimental results show that compared with Yolov5s, the mean average precision of the proposed algorithm is increased from 0.85 to 0.887, and the frame rate is increased from 61 fps to 70 fps.