Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions
 
conference paper

Learning Gaussian Mixtures with Generalized Linear Models: Precise Asymptotics in High-dimensions

Loureiro, Bruno  
•
Sicuro, Gabriele  
•
Gerbelot, Cedric
Show more
2021
Advances in Neural Information Processing Systems 34 pre-proceedings (NeurIPS 2021)
35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. In this manuscript, we characterise the learning of a mixture of K Gaussians with generic means and covariances via empirical risk minimisation (ERM) with any convex loss and regularisation. In particular, we prove exact asymptotics characterising the ERM estimator in high-dimensions, extending several previous results about Gaussian mixture classification in the literature. We exemplify our result in two tasks of interest in statistical learning: a) classification for a mixture with sparse means, where we study the efficiency of ℓ1 penalty with respect to ℓ2; b) max-margin multi-class classification, where we characterise the phase transition on the existence of the multi-class logistic maximum likelihood estimator for K>2. Finally, we discuss how our theory can be applied beyond the scope of synthetic data, showing that in different cases Gaussian mixtures capture closely the learning curve of classification tasks in real data sets.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

Learning_Gaussian_Mixtures_with_Generalised_Linear.pdf

Type

N/a

Access type

openaccess

License Condition

n/a

Size

2.6 MB

Format

Adobe PDF

Checksum (MD5)

9b9c3eab26f03db72c0b0d639c245ae0

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés