The neural network (NN) or artificial neural network (ANN) (McCulloch and Pitts in Bull Math Biol 52(1–2):99–115, 1990; Tickle et al. in IEEE Trans Neural Netw 9(6):1057–1068) in the field of artificial intelligence, database and machine learning is developed and inspired by the biological NNs. NNs have a layer structure, as shown in Fig. 9.1. The first layer is the input layer, the last layer is the output layer, and the layers between the input and output layers are the middle layers, called the hidden layers as well. There are neurons (nodes) in each layer and the numbers of neurons in the input and output layers are equal to the numbers of features and responses, respectively. Typically, synapses link neurons in one layer to neurons in its closest neighboring layer(s). Usually, no synapses link neurons among the same layer and no synapses link neurons in one layer to other layers, rather than its closest neighbor layer. NNs with such topology structure are called the multilayer feedforward NNs, which are introduced here. Neurons in hidden layers are mathematical (logical) operators to generate predictions. A mathematical (logical) operator uses an activation function to implement the mathematical (logical) operation and then passes the results in a weighted manner to the next layer of neurons. Neurons in the output layer gather the prediction and make the final result.