Multi-modal Quality Prediction Algorithm Based on Anomalous Energy Tracking Attention

With the integration of manufacturing and the new generation of information technology, it becomes possible to utilize quality prediction technology instead of inspection technology. This paper focuses on achieving efficient product quality prediction by utilizing various relevant data from the product production process to overcome the limitations of manual quality inspection. Aiming to address the issues related to one-sided information of unimodal data and the close relationship between abnormal fluctuations in the production process and quality problems, a product quality prediction model called MAETAFormer is proposed. This model is based on the attention mechanism of anomalous energy tracking. It enhances the information of anomalous energy through the wavelet transform and a focusing module that targets sparse high-frequency anomalous fluctuations. The model further integrates the anomalous energy using an attention mechanism between the dual streams of time series data and image data. Finally, it deeply fuses the two modalities through the bilinear pooling module. The model is tested on the collected Strand Multimodal Dataset and demonstrates superior performance compared to advanced multimodal networks like LXMERT.