Model of Deep Learning Networks

This section of the book primarily addresses the deep learning model designing and development. Deep learning network is emerging as an another tool set to model given physical process (16,17,18). Observed data of a given physical process is used in design and development of deep learning networks. Probability distribution for a given data set is associated with deep learning networks which represent a given data set. A neural network is used model Boltzmann machine, but training Boltzmann machine is still an open problem. Thus, restricted Boltzmann machine (J. S, Latex document handling, 2018. https://www.overleaf.com/project ) is trending as a way ahead and the same is used in a model neural network, convolutional neural network, etc. The mentioned restricted Boltzmann machine is using Bayes network and data collection is required to support model parameters. Innovation of CNN had resulted in providing a tool set to handle modeling of observed data. Brooks-Iyengar algorithm (J. S, Setting up ai computer (Jetson Nano), 2018. https://github.com/DLinIoTedge/dltrainBook/tree/jk/Tool-Set/jetson/setup ; J. S, IBM watson machine learning: Community edition, 2019. https://github.com/DLinIoTedge/dltrainBook/blob/jk/Tool-Set/Power9cpu/IBM-WML-CE.md ) provides methods and apparatus to solve a special class of Boltzmann machine which is in line with multilayer perceptron (MLP). The design of a deep learning network uses NN, CNN, RNN, etc. to model a network. The development of deep learning networks (N. Kumar, DLtrain source code in C, C \(++\) , 2018. https://github.com/DLinIoTedge/dltrainBook/tree/jk/DLtrain/Ubuntu/C-ConvNov22 ) requires to train NN, CNN, RNN, etc. by using a data set. Finding a probability distribution for a given data is defined as computability problem in the sense of Kolmogorov computability (J. S, Setting up AI computer (Jetson Nano), 2018. https://github.com/DLinIoTedge/dltrainBook/tree/jk/Tool-Set/jetson/setup ). Back propagation is one class of algorithms that leads to suboptimal deep learning networks. Pre-trained deep learning networks become a starting step to train a network with additional data set. Compression (quantization of bias and weights, pruning) of a trained deep learning network also appears to be critical for successful deployment of a trained deep learning network in a given IoT native device or cloud native system. The abovementioned items are discussed in this chapter, but still there is a scope to enhance with a lot more detail in Kolmogorov complexity and also the use of Pontryagin duality (J. S, IBM Watson machine learning: community edition, 2019. https://github.com/DLinIoTedge/dltrainBook/blob/jk/Tool-Set/Power9cpu/IBM-WML-CE.md ; N. Kumar, DLtrain source code in C, C \(++\) , 2018. https://github.com/DLinIoTedge/dltrainBook/tree/jk/DLtrain/Ubuntu/C-ConvNov22 ; J. S, Workflow to build DLtrain to perform inference in x86 with ubuntu OS to train deep learning networks, 2018. https://github.com/DLinIoTedge/dltrainBook/tree/jk/DLtrain/Ubuntu/C-ConvNov22/cpuInfer ) to handle Kolmogorov complexity.