Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Multi-layer state evolution under random convolutional design
 
research article

Multi-layer state evolution under random convolutional design

Daniels, Max
•
Gerbelot, Cedric
•
Krzakala, Florent  
Show more
November 1, 2023
Journal Of Statistical Mechanics-Theory And Experiment

Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generative priors with fully connected layers and Gaussian i.i.d. weights, this was achieved by the multi-layer approximate message (ML-AMP) algorithm via a rigorous state evolution. However, practical generative priors are typically convolutional, allowing for computational benefits and inductive biases, and so the Gaussian i.i.d. weight assumption is very limiting. In this paper, we overcome this limitation and establish the state evolution of ML-AMP for random convolutional layers. We prove in particular that random convolutional layers belong to the same universality class as Gaussian matrices. Our proof technique is of an independent interest as it establishes a mapping between convolutional matrices and spatially coupled sensing matrices used in coding theory.

  • Details
  • Metrics
Type
research article
DOI
10.1088/1742-5468/ad0220
Web of Science ID

WOS:001105355700001

Author(s)
Daniels, Max
Gerbelot, Cedric
Krzakala, Florent  
Zdeborova, Lenka  
Date Issued

2023-11-01

Published in
Journal Of Statistical Mechanics-Theory And Experiment
Volume

2023

Issue

11

Article Number

114002

Subjects

Technology

•

Physical Sciences

•

Cavity And Replica Method

•

Deep Learning

•

Machine Learning

•

Statistical Inference

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IDEPHICS1  
SPOC1  
FunderGrant Number

M D acknowledges funding from Northeastern University's Undergraduate Research amp; Fellowships office and the Goldwater Award. We acknowledge funding from the ERC under the European Union's Horizon 2020 Research and Innovation Program Grant Agreement 714

Northeastern University's Undergraduate Research amp; Fellowships office and the Goldwater Award

714608-SMiLe

ERC under the European Union

Available on Infoscience
February 19, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/204230
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés