In recent years, the architecture and structure of Deep Neural Networks (DNNs) have become progressively more complex in order to respond to the increasing complexity of real problems. A strategy to deal with this complexity when it affects training would be to partition DNN training in some way: for example, by distributing it among different components of a computer network. For this, training (which is in essence the minimization of the loss function) should be performed through separated “smaller pieces”. This paper offers an alternative to the gradient-based DNN training from a Dynamic Programming (DP) point of view (DP is an optimisation methodology supported by the division of a complex problem into many problems of lower complexity). To do so, conditions which enable the DNN minimization algorithm to be solved under a DP perspective are studied here. In this line, in this work is proved that any artificial neural network ANN (and thus also DNNs) with monotonic activation is separable. Furthermore, whenever ANNs are considered as a dynamical system in the form of a network (known as coupled cell networks CCNs), we show that the transmission function is a separable function assuming that the activation is non-decreasing.