Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization
 
conference paper

Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Vlaski, Stefan  
•
Sayed, Ali H.  
January 1, 2020
2020 Ieee International Conference On Acoustics, Speech, And Signal Processing
IEEE International Conference on Acoustics, Speech, and Signal Processing

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to noncooperative approaches in the strongly-convex setting. More recently, these results have been extended to the pursuit of first-order stationary points in non-convex environments. In this work, we examine in detail the dependence of second-order convergence guarantees on the spectral properties of the combination policy for non-convex multi agent optimization. We establish linear speedup in saddle-point escape time in the number of agents for symmetric combination policies and study the potential for further improvement by employing asymmetric combination weights. The results imply that a linear speedup can be expected in the pursuit of second-order stationary points, which exclude local maxima as well as strict saddle-points and correspond to local or even global minima in many important learning settings.

  • Details
  • Metrics
Type
conference paper
DOI
10.1109/ICASSP40776.2020.9054327
Web of Science ID

WOS:000615970408172

Author(s)
Vlaski, Stefan  
Sayed, Ali H.  
Date Issued

2020-01-01

Publisher

IEEE

Publisher place

New York

Published in
2020 Ieee International Conference On Acoustics, Speech, And Signal Processing
ISBN of the book

978-1-5090-6631-5

Series title/Series vol.

International Conference on Acoustics Speech and Signal Processing ICASSP

Start page

8589

End page

8593

Subjects

Acoustics

•

Engineering, Electrical & Electronic

•

Engineering

•

non-convex optimization

•

saddle-point

•

second-order stationarity

•

minima

•

decentralized algorithm

•

centralized algorithm

•

diffusion strategy

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
ASL  
Event nameEvent placeEvent date
IEEE International Conference on Acoustics, Speech, and Signal Processing

Barcelona, SPAIN

May 04-08, 2020

Available on Infoscience
March 26, 2021
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/176188
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés