Learning Effects and Network Decisions

Methods will be presented that explain what a network has learned. The inspection of trained filters continues first attempts in Chap. 8 to investigate what kind of knowledge a filter represents. While filters of the first layers may be interpreted as templates for relevant image textures, filter effects from later layers have to be backpropagated to the input layer first. A deconvolution network is described that does this based on an input image. A more general way to look at trained filters is to search for optimal input of a given filter. An image is computed that maximizes the activation for some filter. A different kind of analysis is to investigate what contributes to a specific decision. Two strategies will be presented. Occlusion analysis investigates the impact of removal of image parts on the classification result. Class activation maps are an alternative to compute what image regions may have been relevant for a labeling decision. They compute the impact of the different feature maps of the last convolution layer on the classification decision and use this as weights to generate a class activation map from weighted averaging over all feature maps.