Faster Coordinate Descent via Adaptive Importance Sampling

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-dual gap estimates and can change at each iteration. We theoretically characterize the performance of our selection rules and demonstrate improvements over the state-of-the-art, and extend our theory and algorithms to general convex objectives. Numerical evidence with hinge-loss support vector machines and Lasso confirm that the practice follows the theory.


Publié dans:
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 54
Présenté à:
20th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017, Fort Lauderdale, Florida, USA, April 20-22, 2017
Année
2017
Publisher:
USA, PMLR
Mots-clefs:
Laboratoires:




 Notice créée le 2017-03-07, modifiée le 2019-09-02

n/a:
284 - Télécharger le documentPDF
284-supp - Télécharger le documentPDF
Lien externe:
Télécharger le documentURL
Évaluer ce document:

Rate this document:
1
2
3
 
(Pas encore évalué)