Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Performance limits of stochastic sub-gradient learning, Part I: Single agent case
 
Loading...
Thumbnail Image
research article

Performance limits of stochastic sub-gradient learning, Part I: Single agent case

Ying, Bicheng
•
Sayed, Ali H.  
2018
Signal Processing

In this work and the supporting Part II [1], we examine the performance of stochastic sub-gradient learning strategies under weaker conditions than usually considered in the literature. The new conditions are shown to be automatically satisfied by several important cases of interest including SVM, LASSO, and Total-Variation denoising formulations. In comparison, these problems do not satisfy the traditional assumptions used in prior analyses and, therefore, conclusions derived from these earlier treatments are not directly applicable to these problems. The results in this article establish that stochastic sub-gradient strategies can attain linear convergence rates, as opposed to sub-linear rates, to the steady-state regime. A realizable exponential-weighting procedure is employed to smooth the intermediate iterates and guarantee useful performance bounds in terms of convergence rate and excessive risk performance. Part I of this work focuses on single-agent scenarios, which are common in stand-alone learning applications, while Part II [1] extends the analysis to networked learners. The theoretical conclusions are illustrated by several examples and simulations, including comparisons with the FISTA procedure.

  • Details
  • Metrics
Type
research article
DOI
10.1016/j.sigpro.2017.10.009
ArXiv ID

1511.07902

Author(s)
Ying, Bicheng
•
Sayed, Ali H.  
Date Issued

2018

Publisher

Elsevier

Published in
Signal Processing
Volume

144

Start page

271

End page

282

Peer reviewed

REVIEWED

Written at

OTHER

EPFL units
ASL  
Available on Infoscience
January 14, 2018
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/143783
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés