Among the existing target detection and feature matching methods, deep learning is the mainstream algorithm at present. However, in complex environments (such as harsh meteorological conditions and changeable geographical environments), the data is usually unreliable, incomplete or inaccurate, which makes the efficiency of target detection and the accuracy of feature matching difficult to meet the actual needs. How to meet the requirement of accurate detection of key parts of the target in the air and sea battlefield under complex environment and feature matching in multiple views is an obstacle, this paper focuses on deep learning and multi-source information fusion technology, and completes the two steps of target hull detection and feature matching of key parts respectively. In terms of target detection, the method which is a ship target recognition algorithm according to YOLOV5 is proposed in this paper, constructs ship infrared image data set, and improves the existing YOLOV5 model with multi-source information fusion technology to form a robust ship detection algorithm, which adopts the method of first rough detection and then fine detection. The multi-scale target fusion mechanism and the small target detection level are integrated to elevate the detection performance of key parts of the target under the influence of complex environment. Through simulation, the detection of ships and key parts is good, which has good engineering significance.