Optimization Methods

Learning algorithms have several parameters such as learning rate, epochs, etc. The output of a learning algorithm is compared to the actual output, and the difference between them is calculated and called the cost function. Optimizers are functions that are used in machine learning algorithms to minimize the difference between the actual output and the predicted output using a gradient. The minimization of the cost function value is carried out by finding the gradient. The gradient is the value change of all the parameters involved in learning with respect to the change happening in the cost function. The slope can be steeper if the gradient is higher, and learning happens in a faster way. This will repeat for several iterations in order to minimize the cost function value. But at the same time, the learning process stops if the slope becomes zero.