The Forward Forward (FF) algorithm has been proposed as a biologically plausible alternative to backpropagation for training deep neural networks. It replaces backward gradient computations with a dual forward pass strategy, where each layer independently optimizes a local “goodness” function to distinguish between positively and negatively labeled data. However, because each layer is trained locally with no global error signals, FF-based training suffers from poor coordination between layers and suboptimal label alignment. In this research, we enhance the Forward Forward framework by using the Hilbert Schmidt Independence Criterion (HSIC) to improve the goodness function at each layer. HSIC serves as a label aware statistical dependence measure, encouraging each layer’s output to retain relevant input structure while aligning more closely with the true class labels. Our formulation introduces distinct HSIC based objectives for positive and negative passes: the positive pass maximizes dependence with the true label, while the negative pass penalizes alignment with incorrect labels. This design maintains the local and backpropagation free nature of FF training while promoting global task coherence and refines its ability to differentiate between positive and negative data more effectively, leading to more robust feature representations and improved learning dynamics. Our experimental results demonstrate that this approach improves the accuracy of the FF algorithm across multiple benchmark datasets, narrowing the performance gap with backpropagation while preserving the FF algorithm’s intrinsic advantages.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing the Forward Forward Algorithm with Label Based Similarity for Improved Neural Network Training

  • Roshan Birjais,
  • Kevin Wang,
  • Waleed Abdulla

摘要

The Forward Forward (FF) algorithm has been proposed as a biologically plausible alternative to backpropagation for training deep neural networks. It replaces backward gradient computations with a dual forward pass strategy, where each layer independently optimizes a local “goodness” function to distinguish between positively and negatively labeled data. However, because each layer is trained locally with no global error signals, FF-based training suffers from poor coordination between layers and suboptimal label alignment. In this research, we enhance the Forward Forward framework by using the Hilbert Schmidt Independence Criterion (HSIC) to improve the goodness function at each layer. HSIC serves as a label aware statistical dependence measure, encouraging each layer’s output to retain relevant input structure while aligning more closely with the true class labels. Our formulation introduces distinct HSIC based objectives for positive and negative passes: the positive pass maximizes dependence with the true label, while the negative pass penalizes alignment with incorrect labels. This design maintains the local and backpropagation free nature of FF training while promoting global task coherence and refines its ability to differentiate between positive and negative data more effectively, leading to more robust feature representations and improved learning dynamics. Our experimental results demonstrate that this approach improves the accuracy of the FF algorithm across multiple benchmark datasets, narrowing the performance gap with backpropagation while preserving the FF algorithm’s intrinsic advantages.