Reinforcement Learning takes an active perspective on the learning of internal models, focusing on an agent acting in an environment. The agent’s control system is tasked with selecting actions for the agent based on the current sensed state. In Reinforcement Learning, the control system is optimized to gain high rewards over time. While the optimization is driven by a usually external reward signal, the process is explorative, as the agent tries to find areas within the environment that offer high rewards. In Deep Reinforcement Learning (DRL), neural networks are employed as function approximators to deal with continuous sensory input spaces. Considering actions as the output of the control system, the exploration of large action spaces quickly requires a high number of exploratory interactions or, even worse, becomes intractable. In this chapter, we will introduce Reinforcement Learning, its conceptual formulation as a Markov Decision Process, and highlight current problems of DRL as well as biologically-inspired approaches that address these issues.