Feature Extraction by Convolutional Neural Network

An optimal mapping from pixel values to class labels by a MLP requires too many nodes and layers for successful training. Convolutional neural networks extend the concept by adding a different type of layer that restricts network expressivity without compromising classifier performance. Part of the network is a sequence of convolutional building blocks that extracts features from images which are submitted to fully connected layers for classification. Feature extraction in a convolutional building block happens by feeding the input through a sequence of convolution layers followed by a pooling step. Convolution layers are applied to spatially organized maps and learn a set of convolution kernels followed by a non-linear activation function to extract intermediate features from the input map. Nodes in a convolution layer are connected to a fixed neighborhood of input nodes and weights of connections to the neighborhood are equal for all output nodes. The pooling layer reduces the spatial resolution of the feature map. The resulting network is deeper than the MLP but adds fewer weights per layer. We describe the effects of a deeper network on training and conclude with an initial experiment with a trained network.