Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers
 
conference paper not in proceedings

The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers

Latorre, Fabian  
•
Dadi, Leello Tadesse  
•
Rolland, Paul Thierry Yves  
Show more
2021
35th Conference on Neural Information Processing Systems (NeurIPS 2021)

It has been recently observed that neural networks, unlike kernel methods, enjoy a reduced sample complexity when the distribution is isotropic (i.e., when the covariance matrix is the identity). We find that this sensitivity to the data distribution is not exclusive to neural networks, and the same phenomenon can be observed on the class of quadratic classifiers (i.e., the sign of a quadratic polynomial) with a nuclear-norm constraint. We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution. Our result improves the dependence on the dimension over the best previously known bound and precisely quantifies the relation between the sample complexity and the level of isotropy of the distribution.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

quadpoly.pdf

Type

Postprint

Version

Accepted version

Access type

openaccess

License Condition

Copyright

Size

357.62 KB

Format

Adobe PDF

Checksum (MD5)

5a5d29835ab5e32e02c7da1bfff01f5a

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés