7 Multi-Layer Perceptron for Image Classification

Artificial neural networks are trainable models that model arbitrarily complex decision boundaries for image classification. A network consists of nodes and edges represented as graph. Networks for image classification are cycle-free and information flows from an input layer through the nodes along the edges to an output layer. The input layer receives features extracted from an image producing a prediction at the output layer. A simple network of this kind is the single-layer perceptron which receives features extracted from an image and carries out a binary classification. We will show that it solves the classification problem by linear logistic regression. Adding layers to the network extends the trainable model to non-linear decision boundaries. We will explain why a single extra layer is sufficient to train an arbitrary mapping from input to output. It makes a multi-layer perceptron a universal solver that can find an optimal non-linear mapping for multinomial logistic regression. We will explain why adding further layers increases the efficiency of such networks. Parameter optimization in the network is done by the backpropagation algorithm that carries out a gradient descent on the network parameters. Finally, the Adam optimizer will be presented as a popular gradient descent method.