Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers
 
conference paper not in proceedings

The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers

Latorre, Fabian  
•
Dadi, Leello Tadesse  
•
Rolland, Paul Thierry Yves  
Show more
2021
35th Conference on Neural Information Processing Systems (NeurIPS 2021)

It has been recently observed that neural networks, unlike kernel methods, enjoy a reduced sample complexity when the distribution is isotropic (i.e., when the covariance matrix is the identity). We find that this sensitivity to the data distribution is not exclusive to neural networks, and the same phenomenon can be observed on the class of quadratic classifiers (i.e., the sign of a quadratic polynomial) with a nuclear-norm constraint. We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution. Our result improves the dependence on the dimension over the best previously known bound and precisely quantifies the relation between the sample complexity and the level of isotropy of the distribution.

  • Files
  • Details
  • Metrics
Type
conference paper not in proceedings
Author(s)
Latorre, Fabian  
Dadi, Leello Tadesse  
Rolland, Paul Thierry Yves  
Cevher, Volkan
Date Issued

2021

Subjects

ml-ai

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LIONS  
Event nameEvent placeEvent date
35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Sydney, Australia

December 6-14, 2021

Available on Infoscience
October 31, 2021
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/182609
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés