Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Extra Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
 
conference paper not in proceedings

Extra Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

Antonakopoulos, Kimon  
•
Kavis, Ali  
•
Cevher, Volkan  orcid-logo
2022
36th Conference on Neural Information Processing Systems (NeurIPS)

This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions. Our algorithm achieves O(σ/T‾‾√) convergence when the oracle feedback is stochastic with variance σ2, and improves its convergence to O(1/T3) with deterministic oracles, where T is the number of iterations. Our method also interpolates these rates without knowing the nature of the oracle apriori, which is enabled by a parameter-free adaptive step-size that is oblivious to the knowledge of smoothness modulus, variance bounds and the diameter of the constrained set. To our knowledge, this is the first universal algorithm with such global guarantees within the second-order optimization literature.

  • Files
  • Details
  • Metrics
Type
conference paper not in proceedings
Author(s)
Antonakopoulos, Kimon  
Kavis, Ali  
Cevher, Volkan  orcid-logo
Date Issued

2022

Total of pages

32

Subjects

ML-AI

Written at

EPFL

EPFL units
LIONS  
Event nameEvent placeEvent date
36th Conference on Neural Information Processing Systems (NeurIPS)

New Orleans, Louisianna, USA

November 28-December 9, 2022

Available on Infoscience
February 20, 2023
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/194938
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés