Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization
 
conference paper not in proceedings

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Kavis, Ali  
•
Skoulakis, Efstratios Panteleimon  
•
Antonakopoulos, Kimon  
Show more
2022
36th Conference on Neural Information Processing Systems (NeurIPS)

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fairly distinct, adaptive step-size schedule with the recursive stochastic path integrated estimator proposed in [Fang et al., 2018]. To our knowledge, Adaspider is the first parameter-free non-convex variance-reduction method in the sense that it does not require the knowledge of problem-dependent parameters, such as smoothness constant L, target accuracy ϵ or any bound on gradient norms. In doing so, we are able to compute an ϵ-stationary point with Õ (n+n‾√/ϵ2) oracle-calls, which matches the respective lower bound up to logarithmic factors.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization.pdf

Type

Postprint

Version

http://purl.org/coar/version/c_ab4af688f83e57aa

Access type

openaccess

License Condition

copyright

Size

563.98 KB

Format

Adobe PDF

Checksum (MD5)

8b9b7deed2207e75f01354df7fdc54d4

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés