Implementing Reversible Neural Networks
摘要
Training a multilayered neural network is a process of passing the training data through the network, calculating the error between predicted and actual outputs, and using backpropagation to adjust the network’s weights to minimize the overall error. This process continues until the network achieves acceptable accuracy. During training, the activation and gradient data for each layer must be stored in memory. An alternative approach to this is to recompute the necessary activation and gradient data instead of storing it. A way to achieve this, Recalculation via Function and Matrix Inverse, is implemented in this work. We can quantify that this saves memory and discuss the relationship with increased calculation costs.