This chapter introduces the fundamental concepts of optimization, particularly in the context of machine learning, and explores the role of derivative-free optimization (DFO) in solving complex computational tasks. Optimization is essential for finding optimal solutions within a solution space, and machine learning often involves formulating such problems to learn generalizable models from data. The chapter highlights the importance of DFO, which does not require gradient information and is suitable for problems with discontinuous or non-differentiable objective functions. It outlines the structure of DFO algorithms, their development, and their application in automatic machine learning (AutoML), where they help automate the selection of algorithms and hyper-parameters. The chapter concludes by presenting the organization of the book, which aims to build theoretical foundations for DFO and design practical algorithms for machine learning tasks.