Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case
 
research article

Performance limits of stochastic sub-gradient learning, Part II: Multi-agent case

Ying, Bicheng
•
Sayed, Ali H.  
2018
Signal Processing

The analysis in Part I [1] revealed interesting properties for subgradient learning algorithms in the context of stochastic optimization. These algorithms are used when the risk functions are non-smooth or involve non-differentiable components. They have been long recognized as being slow converging methods. However, it was revealed in Part I [1] that the rate of convergence becomes linear for stochastic optimization problems, with the error iterate converging at an exponential rate αi to within an neighborhood of the optimizer, for some α ∈ (0, 1) and small step-size μ. The conclusion was established under weaker assumptions than the prior literature and, moreover, several important problems were shown to satisfy these weaker assumptions automatically. These results revealed that sub-gradient learning methods have more favorable behavior than originally thought. The results of Part I [1] were exclusive to single-agent adaptation. The purpose of current Part II is to examine the implications of these discoveries when a collection of networked agents employs subgradient learning as their cooperative mechanism. The analysis will show that, despite the coupled dynamics that arises in a networked scenario, the agents are still able to attain linear convergence in the stochastic case; they are also able to reach agreement within O(μ) of the optimizer.

  • Details
  • Metrics
Type
research article
DOI
10.1016/j.sigpro.2017.10.007
ArXiv ID

1704.06025

Author(s)
Ying, Bicheng
Sayed, Ali H.  
Date Issued

2018

Publisher

Elsevier

Published in
Signal Processing
Volume

144

Start page

253

End page

264

Editorial or Peer reviewed

REVIEWED

Written at

OTHER

EPFL units
ASL  
Available on Infoscience
December 21, 2017
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/143533
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés