Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
 
conference paper

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Sarao Mannelli, Stefano
•
Biroli, Giulio
•
Cammarota, Chiara
Show more
2020
Proceeding of the 2020 Advances in Neural Information Processing Systems
Advances in Neural Information Processing Systems

Despite the widespread use of gradient-based algorithms for optimizing high-dimensional non-convex functions, understanding their ability of finding good minima instead of being trapped in spurious ones remains to a large extent an open problem. Here we focus on gradient flow dynamics for phase retrieval from random measurements. When the ratio of the number of measurements over the input dimension is small the dynamics remains trapped in spurious minima with large basins of attraction. We find analytically that above a critical ratio those critical points become unstable developing a negative direction toward the signal. By numerical experiments we show that in this regime the gradient flow algorithm is not trapped; it drifts away from the spurious critical points along the unstable direction and succeeds in finding the global minimum. Using tools from statistical physics we characterize this phenomenon, which is related to a BBP-type transition in the Hessian of the spurious minima.

  • Details
  • Metrics
Type
conference paper
Author(s)
Sarao Mannelli, Stefano
Biroli, Giulio
Cammarota, Chiara
Krzakala, Florent  
Urbani, Pierfrancesco
Zdeborová, Lenka  
Date Issued

2020

Publisher

Curran Associates, Inc.

Published in
Proceeding of the 2020 Advances in Neural Information Processing Systems
Total of pages

9

Series title/Series vol.

Advances in Neural Information Processing Systems; 33

Volume

33

Start page

3265

URL

paper

https://papers.nips.cc/paper/2020/file/2172fde49301047270b2897085e4319d-Paper.pdf
Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IDEPHICS1  
IDEPHICS2  
SPOC1  
Show more
Event nameEvent date
Advances in Neural Information Processing Systems

Dec 6, 2020 – Dec 12, 2020

Available on Infoscience
March 5, 2021
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/175772
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés