Training a multilayered neural network is a process of passing the training data through the network, calculating the error between predicted and actual outputs, and using backpropagation to adjust the network’s weights to minimize the overall error. This process continues until the network achieves acceptable accuracy. During training, the activation and gradient data for each layer must be stored in memory. An alternative approach to this is to recompute the necessary activation and gradient data instead of storing it. A way to achieve this, Recalculation via Function and Matrix Inverse, is implemented in this work. We can quantify that this saves memory and discuss the relationship with increased calculation costs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Implementing Reversible Neural Networks

  • Uku Zingel,
  • Stefan Kuhn,
  • Eero Vainikko

摘要

Training a multilayered neural network is a process of passing the training data through the network, calculating the error between predicted and actual outputs, and using backpropagation to adjust the network’s weights to minimize the overall error. This process continues until the network achieves acceptable accuracy. During training, the activation and gradient data for each layer must be stored in memory. An alternative approach to this is to recompute the necessary activation and gradient data instead of storing it. A way to achieve this, Recalculation via Function and Matrix Inverse, is implemented in this work. We can quantify that this saves memory and discuss the relationship with increased calculation costs.