Infrared sensors are critical equipment for monitoring substations, but they are subjected to complex multi-physical field stresses in substation applications, leading to a significantly higher failure rate of infrared sensors in substations compared to other applications. Effective fault diagnosis of infrared sensors is of great importance for improving the safety of substations. This article proposes a method for fault diagnosis of infrared sensors. First, the network weights are trained based on the ImageNet dataset and kept fixed. Then, a transfer learning approach is used to fine-tune the model based on a dataset of infrared sensor failures, resulting in the classification results. To verify the effectiveness of the proposed method, the improved transfer learning model is compared to the model before improvement. Experimental results show that the proposed method greatly reduces training time and improves classification accuracy compared to the previous model. In conclusion, the method presented in this article offers a new approach for fault diagnosis of infrared sensors, which can contribute to enhancing the reliability of infrared sensors in substations.