Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs
 
conference paper

A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs

Tanner, Kasimir
•
Vilucchio, Matteo  
•
Loureiro, Bruno  
Show more
May 3, 2025
Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS) [Forthcoming publication]
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2025)

This work investigates adversarial training in the context of margin-based linear classifiers in the high-dimensional regime where the dimension d and the number of data points n diverge with a fixed ratio α = n/d. We introduce a tractable mathematical model where the interplay between the data and adversarial attacker geometries can be studied, while capturing the core phenomenology observed in the adversarial robustness literature. Our main theoretical contribution is an exact asymptotic description of the sufficient statistics for the adversarial empirical risk minimiser, under generic convex and non-increasing losses for a Block Feature Model. Our result allow us to precisely characterise which directions in the data are associated with a higher generalisation/robustness trade-off, as defined by a robustness and a usefulness metric. We show that the the presence of multiple different feature types is crucial to the high sample complexity performances of adversarial training. In particular, we unveil the existence of directions which can be defended without penalising accuracy. Finally, we show the advantage of defending non-robust features during training, identifying a uniform protection as an inherently effective defence mechanism.

  • Files
  • Details
  • Metrics
Type
conference paper
ArXiv ID

2402.05674

Author(s)
Tanner, Kasimir
Vilucchio, Matteo  

EPFL

Loureiro, Bruno  

École Normale Supérieure - PSL

Krzakala, Florent  

EPFL

Date Issued

2025-05-03

Published in
Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS) [Forthcoming publication]
Subjects

Adversarial Regression

•

Structured Data

•

AMP

•

Linear Regression

•

GLMs

•

ERM

•

high-dimensional statistics

•

learning theory

URL

ArXiv

https://doi.org/10.48550/arXiv.2402.05674
Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IDEPHICS1  
Event nameEvent acronymEvent placeEvent date
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2025)

AISTATS 2025

Mai Khao, Thailand

2025-05-03 - 2025-05-05

Available on Infoscience
April 14, 2025
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/249141
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés