Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Leveraging Sparsity for Sample-Efficient Preference Learning: A Theoretical Perspective
 
conference paper

Leveraging Sparsity for Sample-Efficient Preference Learning: A Theoretical Perspective

Yao, Yunzhen  
•
He, Lie  
•
Gastpar, Michael  
July 2025
Proceedings of the 42nd International Conference on Machine Learning
42nd International Conference on Machine Learning, ICML 2025

This paper considers the sample-efficiency of preference learning, which models and predicts human choices based on comparative judgments. The minimax optimal estimation error rate Θ(d/n) in classical estimation theory requires that the number of samples n scales linearly with the dimensionality of the feature space d. However, the high dimensionality of the feature space and the high cost of collecting human-annotated data challenge the efficiency of traditional estimation methods. To remedy this, we leverage sparsity in the preference model and establish sharp error rates. We show that under the sparse random utility model, where the parameter of the reward function is k-sparse, the minimax optimal rate can be reduced to Θ(k/nlog(d/k)). Furthermore, we analyze the ℓ1-regularized estimator and show that it achieves near-optimal rate under mild assumptions on the Gram matrix. Experiments on synthetic data and LLM alignment data validate our theoretical findings, showing that sparsity-aware methods significantly reduce sample complexity and improve prediction accuracy.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

2501.18282v4.pdf

Type

Main Document

Version

Published version

Access type

openaccess

License Condition

CC BY

Size

765.53 KB

Format

Adobe PDF

Checksum (MD5)

92a2e015f033ef896959b355ca05acae

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés