Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. The effect of smooth parametrizations on nonconvex optimization landscapes
 
research article

The effect of smooth parametrizations on nonconvex optimization landscapes

Levin, Eitan
•
Kileel, Joe
•
Boumal, Nicolas  
March 4, 2024
Mathematical Programming

We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good guarantees) or for theoretical purposes (e.g., to reveal that the landscape satisfies a strict saddle property). In both cases, the central question is: how do the landscapes of the two problems relate? More precisely: how do desirable points such as local minima and critical points in one problem relate to those in the other problem? A key finding in this paper is that these relations are often determined by the parametrization itself, and are almost entirely independent of the cost function. Accordingly, we introduce a general framework to study parametrizations by their effect on landscapes. The framework enables us to obtain new guarantees for an array of problems, some of which were previously treated on a case-by-case basis in the literature. Applications include: optimizing low-rank matrices and tensors through factorizations; solving semidefinite programs via the Burer-Monteiro approach; training neural networks by optimizing their weights and biases; and quotienting out symmetries.

  • Files
  • Details
  • Metrics
Type
research article
DOI
10.1007/s10107-024-02058-3
Web of Science ID

WOS:001173504400002

Author(s)
Levin, Eitan
Kileel, Joe
Boumal, Nicolas  
Date Issued

2024-03-04

Publisher

Springer Heidelberg

Published in
Mathematical Programming
Subjects

Technology

•

Physical Sciences

•

Overparametrization

•

Symmetries In Landscapes

•

Benign Nonconvexity

•

Strict Saddles

•

Low Rank Optimization

•

Hadamard Parametrization Of Simplex

•

Optimization On Manifolds

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
OPTIM  
FunderGrant Number

EPFL Lausanne

MB22.00027

Swiss State Secretariat for Education, Research and Innovation (SERI)

CISE-IIS 2312746

NSF

Available on Infoscience
May 1, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/207583
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés