<p>Transformer models excel in modeling long-distance dependencies but have certain limitations in extracting non-stationary trend information, which often leads to prediction bias. This is especially problematic in complex industrial processes, where small fluctuations are frequently misidentified as noise, thereby weakening the model’s ability to capture local dynamic features. Additionally, when dealing with multivariate problems, failing to account for the differences between variables can exacerbate prediction bias. To address these issues, this paper proposes a cointegration dynamic multi-embedding Transformer (CDME-Transformer) model. The model first designs a Cointegration Analysis Layer (CALayer) to extract the stationary features of the data, enhancing the model’s robustness to non-stationary signals. Secondly, by embedding both endogenous (the variables of primary interest in process monitoring) and exogenous variables (auxiliary factors that can provide valuable information for interpreting endogenous variables) and combining self-attention and cross-attention mechanisms, it improves the model’s ability to capture interactions between endogenous and exogenous variables. Finally, the model incorporates a dynamic distance metric to reduce the impact of data fluctuations on the detection results, thereby accurately capturing the temporal dynamics of industrial processes and performing anomaly detection based on prediction bias. Experimental results from penicillin fermentation and actual E. coli fermentation data demonstrated that the CDME-Transformer model outperformed existing methods by a significant margin.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fault Detection for Complex Industrial Processes Based on Cointegration Dynamic Multi-embedding Transformer

  • Xuejin Gao,
  • Wenxuan Ma,
  • Huayun Han,
  • Huihui Gao,
  • Yongsheng Qi

摘要

Transformer models excel in modeling long-distance dependencies but have certain limitations in extracting non-stationary trend information, which often leads to prediction bias. This is especially problematic in complex industrial processes, where small fluctuations are frequently misidentified as noise, thereby weakening the model’s ability to capture local dynamic features. Additionally, when dealing with multivariate problems, failing to account for the differences between variables can exacerbate prediction bias. To address these issues, this paper proposes a cointegration dynamic multi-embedding Transformer (CDME-Transformer) model. The model first designs a Cointegration Analysis Layer (CALayer) to extract the stationary features of the data, enhancing the model’s robustness to non-stationary signals. Secondly, by embedding both endogenous (the variables of primary interest in process monitoring) and exogenous variables (auxiliary factors that can provide valuable information for interpreting endogenous variables) and combining self-attention and cross-attention mechanisms, it improves the model’s ability to capture interactions between endogenous and exogenous variables. Finally, the model incorporates a dynamic distance metric to reduce the impact of data fluctuations on the detection results, thereby accurately capturing the temporal dynamics of industrial processes and performing anomaly detection based on prediction bias. Experimental results from penicillin fermentation and actual E. coli fermentation data demonstrated that the CDME-Transformer model outperformed existing methods by a significant margin.