Distributed Dual Proximal Gradient Algorithms for Nonsmooth Constrained Composite Optimization
摘要
Distributed dual methods play significant roles in large-scale optimization, which effectively resolve many constrained optimization problems in machine learning and power systems. In this chapter, we focus on studying a class of totally nonsmooth constrained composite optimization problems over multi-agent systems, where the mutual goal of agents in the system is to optimize a sum of two separable nonsmooth functions consisting of a strongly-convex function and another convex (not necessarily strongly-convex) function. Agents in the system conduct parallel local computation and communication in the overall process without leaking their private information. In order to resolve the totally nonsmooth constrained composite optimization problem in a fully distributed manner, we devise a synchronous distributed dual proximal (SynDe-DuPro) gradient algorithm and its asynchronous version (AsynDe-DuPro) based on the randomized block-coordinate method. Both SynDe-DuPro and AsynDe-DuPro algorithms are theoretically proved to achieve the globally optimal solution to the totally nonsmooth constrained composite optimization problem relied on the quasi-Fejér monotone theorem. As a main result, AsynDe-DuPro algorithm attains the globally optimal solution without requiring all agents to be activated at each iteration and thus is more robust than most existing synchronous algorithms. The practicability of the proposed algorithms and correctness of the theoretical findings are demonstrated by the experiments on a constrained Distributed Sparse Logistic Regression (DSLR) problem in machine learning and a Distributed Energy Resources Coordination (DERC) problem in power systems.