Constrained Optimization and Duality
摘要
In many machine learning models, such as nonnegative regression and box regression, the optimization variables are constrained. Therefore, one needs to find an optimal solution only over a specific region of the optimization space, referred to as the feasible region in optimization parlance. The straightforward use of a gradient-descent procedure does not work, because an unconstrained step might move the optimization variables outside the feasible region.