Vector Analysis of Deep Neural Network Training Process

The simplicity of the idea of gradient descent, commonly referred to as “backpropagation,” and the successes of neural network algorithms in solving intelligent tasks, on the one hand, create the impression of the correctness of its widespread application, while on the other hand, they complicate the understanding of the mechanisms by which neural networks implement complex transformations of input signals into outputs. Advancing understanding of neural network algorithms’ operation may be facilitated by vector analysis of the deep neural network learning process. This article provides several examples of using vector-matrix analysis for various aspects of neural network training based on the backpropagation method. They not only help to better understand the essence of the transformations performed but also offer recommendations for accelerating and “harmonizing” the neural network training process. However, the aim of the article is not to improve individual aspects of neural network algorithm operation but to demonstrate the effectiveness of applying vector-matrix analysis to study various properties of neural network data processing.