A Gentle Introduction to Deep Learning

The so-called “first AI winter” lasted from the early 1970s until 1980, during which only very limited research activities took place. The main reason was the drastic reduction of research funding, which resulted from the general disappointment about the severe limitations of the early perceptron and simple neural network approaches described in Chap. 17. This chapter introduces the relevant concepts and derives important method formulations that were required to accelerate all of the developments leading to today’s deep learning. Among those are new types of activation functions, the concept of backpropagation, and techniques such as training with mini-batches. This is then implemented as Python code and used for machine learning experiments, which help to understand the introduced theoretical concepts and methods.