Convergence of the Generalized Alternating Direction Method of Multipliers for a Class of Nonconvex Optimization Problems
-
摘要: 考虑利用广义交替方向法(GADMM)求解线性约束两个函数和的最小值问题,其中一个函数为凸函数,另一个函数可以表示为两个凸函数的差.对GADMM的每一个子问题,采用两个凸函数之差算法中的线性化技术来处理.通过假定相应函数满足Kurdyka-ojasiewicz不等式,当增广Lagrange(拉格朗日)函数的罚参数充分大时,证明了GADMM所产生的迭代序列收敛到增广Lagrange函数的稳定点.最后,给出了该算法的收敛速度分析.
-
关键词:
- 广义交替方向法 /
- Kurdyka-ojasiewicz不等式 /
- 非凸优化 /
- 收敛性
Abstract: The generalized alternating direction method of multipliers (GADMM) for the minimization problems of the sum of 2 functions with linear constraints was considered, where one function was convex and the other can be expressed as the difference of 2 convex functions. For each subproblem in the GADMM, the linearization technique in the convex function difference algorithm was employed. Under the assumptions that the associated functions satisfy the Kurdyka-ojasiewicz inequality, the sequence generated with the GADMM converges to a critical point of the augmented Lagrangian function, while the penalty parameter in the augmented Lagrangian function is sufficiently large. At last, the convergence rate of the algorithm was established. -
[1] GLOWINSKI R, MARROCCO A. Sur l’approximation par éléments finis d’ordre un et la résolution par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires[J]. Journal of Equine Veterinary Science,1975,2: 41-76. [2] YANG J F, YUAN X M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization[J]. Mathematics of Computation,2013,82(281): 301-329. [3] HE B S, YUAN X M. On non-ergodic convergence rate of douglas-rachford alternating direction method of multipliers[J]. Numerische Mathematik,2015,130(3): 567-577. [4] HE B S, YUAN X M. On the o(1/n) convergence rate of the Douglas-Rachford alternating direction method[J]. Society for Industrial and Applied Mathematics,2012,50(2): 700-709. [5] HONG M Y, LUO Z Q. On the linear convergence of the alternating direction method of multipliers[J]. Mathematical Programming,2017,162(2): 165-199. [6] GUO K, HAN D R, WU T T. Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints[J]. International Journal of Computer Mathematic,2017,94(8): 1653-1669. [7] GUO K, HAN D R, WANG Z W, et al. Convergence of ADMM for multi-block nonconvex separable optimization models[J]. Frontiers of Mathematics in China,2017,12(5): 1139-1162. [8] LI G Y, PONG T K. Global convergence of splitting methods for nonconvex composite optimization[J]. SIAM Journal on Optimization,2015,25(4): 2434-2460. [9] YU W, YIN W T, ZENG J S. Global convergence of ADMM in nonconvex nonsmooth optimization[J/OL]. Journal of Scientific Computing,2015. [2017-12-27]. https: // doi.org/ 10.1007/s10915-018-0757-z. [10] GABAY D. Applications of the method of multipliers tovariational inequalities[C]// In Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Amsterdam, Holland, 1983. [11] LIONS P L, MERCIER B. Splitting algorithms for the sum of two nonlinear operators[J]. Siam Journal on Numerical Analysis,1979,16(6): 964-979. [12] ECKSTEIN J, BERTSEKAS D. On the Douglas-Rachford splitting method and proximal point algorithm for maximal monotone operators[J]. Mathematical Programming,1992,55(1): 293-318. [13] MARTIENT B. Regularisation d’inéquations variations par approximations succesives[J]. Rev Franaise Informat Recherche Opérationnelle,1970,4(4): 154-159. [14] YIN P A, XIN J. Iterative l1-minimization for non-convex compressed sensing[J]. Journal of Computational Mathematics,2017,35(4): 439-451. [15] LOU Y F, YIN P H, XIN J. Point source super-resolution via non-convex l1-based methods[J]. Journal of Scientific Computing,2016,68(3): 1082-1100. [16] YIN P H, LOU Y F, HE Q, et al. Minimization of 1-2 for compressed sensing[J]. SIAM Journal on Scientific Computing,2015,37(1): A536-A563. [17] SUN T, YIN P H, CHENG L Z, et al. Alternating direction method of multipliers with difference of convex functions[J]. Advances in Computational Mathematics,2017,44(3): 1-22. [18] ROCKAFELLAR R T, WETS R J. Variational Analysis [M]. Berlin: Springer-verlag, 1998. [19] ROCKAFELLAR R T. Convex Analysis [M]. Princeton: Princeton University Press, 1970. [20] ATTOUCH H, BOLTE J, REDONT P, et al. Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-ojasiewicz inequality[J]. Mathematics of Operations Research,2010,35(2): 438-457. [21] BOLTE J, SABACH S, TEBOULLE M. Proximal alternatinglinearized minimization for nonconvex and nonsmooth problems[J]. Mathematical Programming,2014,146(1): 459-494. [22] NESTEROV Y. Introductory Lectures on Convex Optimization: A Basic Course [M]. Boston: Kluwer Academic Publishers, 2004. [23] ATTOUCH H, BOLTE J. On the convergence of the proximal algorithm for nonsmooth functions involving analytic features[J]. Mathematical Programming,2009,116(1): 5-16.
点击查看大图
计量
- 文章访问数: 1016
- HTML全文浏览量: 151
- PDF下载量: 867
- 被引次数: 0