This chapter presents essential knowledge about the Artificial Neural Network (ANN), or more specifically, shallow ANNs. We will first check the structure of biological neurons to understand how ANN can mathematically imitate biological NN’s behavior. Next, the mathematical formulation of the well-known M-P neuron model and other significant concepts, such as activation function, will be discussed. Then, we will see how perceptron, as one of the simplest and most typical NNs, works from a mathematical perspective. In particular, the training of such a network via backpropagation will be detailed. The yielded equations will be generalized for NNs with more than three layers to guide implementations. Based on that, extra skills about NN implementations, the illustration of the implementation procedure with an example, and the transformation of data as it moves through a network, will be discussed in detail. Finally, a few other shallow ANN issues will be briefly mentioned.