We generalize the shallow neural network introduced in the previous chapter to a deep neural network. In mathematical terms, a deep neural network can be viewed as a nested composite function comprising multiple layers of intermediate (matrix-valued) variables and several parameter matrices that define the functions connecting consecutive layers. The gradient of the objective function with respect to each parameter matrix is obtained through a straightforward application of the chain rule in multivariate calculus. This process, which proceeds in the reverse order of the forward computation of the objective function, is known as backward propagation.