Neural Networks and Deep Learning

Artificial neural networks (ANNs) inspire biological networks as powerful artificial intelligence tools. ANN is an object that imitates the neural network constituting the human brain so that the computer can learn and make decisions like a human. On the other hand, deep learning indicates a neural network with more than three layers. Deep neural networks are capable of extracting higher-level features from the raw data to solve complicated optimization problems. In this chapter, we intend to present a summary of deep neural networks and review their background. Then, the activation functions and concept of parameter selection in deep learning are discussed. In the end, the performance of deep neural models is presented, and classic deep learning models, including stacking automatic encoders, convolutional neural networks, deep probabilistic neural networks, and generative adversarial networks, are introduced.