Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Approximation of Lipschitz Functions Using Deep Spline Neural Networks
 
research article

Approximation of Lipschitz Functions Using Deep Spline Neural Networks

Neumayer, Sebastian  
•
Goujon, Alexis  
•
Bohra, Pakshal  
Show more
January 1, 2023
Siam Journal On Mathematics Of Data Science

Although Lipschitz-constrained neural networks have many applications in machine learning, the design and training of expressive Lipschitz-constrained networks is very challenging. Since the popular rectified linear-unit networks have provable disadvantages in this setting, we propose using learnable spline activation functions with at least three linear regions instead. We prove that our choice is universal among all componentwise 1-Lipschitz activation functions in the sense that no other weight-constrained architecture can approximate a larger class of functions. Additionally, our choice is at least as expressive as the recently introduced non-componentwise Groupsort activation function for spectral-norm-constrained weights. The theoretical findings of this paper are consistent with previously published numerical results.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

neumayer2301.pdf

Type

Publisher

Version

Published version

Access type

openaccess

License Condition

CC BY-NC-ND

Size

459.96 KB

Format

Adobe PDF

Checksum (MD5)

2af43c8e9c046d5dbb1dd36dd398875c

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés