Learning Optimal Policies

Reinforcement learning is an area of Machine Learning in which a software program (agent) must select an action at each time step with the goal of achieving the highest possible sum of rewards over time. An action is determined based on the current state and affects the reward, often many time steps later. Examples applications include games, robotic controls, and self-driving cars. Deep neural networks (DNNs) can be used to assign a sum of expected rewards to a state-action pair. DNNs are particularly suitable because they can approximate the underlying functions well and may be employed to select the best action for a state. Q-Networks predict an expected reward sum for each state-action pair. A stochastic policy is suitable for decision situations with random influences and determines for each state an optimal probability distribution over the possible actions. For both types of models, training procedures are derived which determine the gradient from a number of simulated model runs. In contrast to previous DNN, the training data is generated during training using a simulated or real environment. Finally, application areas of reinforcement learning are described, such as video games, robot control, and autonomous vehicles.