Efficient Lightweight Network with Transformer-Based Distillation for Micro-crack Detection of Solar Cells

Micro-cracks on solar cells often affect the power generation efficiency, so this paper proposes a lightweight network for cell image micro-crack detection task. Firstly, a Feature Selection framework is proposed, which can efficiently and adaptively decide the number of layers of the feature extraction network, and clip unnecessary feature generation process. In addition, based on the design of the Transformer layer, Transformer Distillation is proposed. In Transformer Distillation, the designed Transformer Refine module excavates the distillation information from the two dimensions of features and relations. Using a combination of Feature Selection and Transformer Distillation, the lightweight networks based on ResNet and ViT can achieve much better effects than the original networks, with classification accuracy rates of 88.58% and 89.35% respectively.