Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression
 
conference paper

TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression

Shukla, Megh  
•
Salzmann, Mathieu  
•
Alahi, Alexandre  
July 21, 2024
Proceedings of the 41st International Conference on Machine Learning (ICML) 2024
41st International Conference on Machine Learning (ICML) 2024

Deep heteroscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. However, recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. While the literature addresses this by proposing alternate formulations to mitigate the impact of the predicted covariance, we focus on improving this predicted covariance itself. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? (2) In the absence of supervision, how can we quantify the accuracy of covariance estimation? We address (1) with a Taylor Induced Covariance (TIC), which captures the randomness of the predicted mean by incorporating its gradient and curvature through the second order Taylor polynomial. Furthermore, we tackle (2) by introducing the Task Agnostic Correlations (TAC) metric, which combines the notion of correlations and absolute error to evaluate the covariance. We evaluate TIC-TAC across multiple experiments spanning synthetic and real-world datasets. Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of negative log-likelihood. We make our code available at: https://github.com/vita-epfl/TIC-TAC

  • Files
  • Details
  • Metrics
Type
conference paper
Author(s)
Shukla, Megh  
Salzmann, Mathieu  
Alahi, Alexandre  
Date Issued

2024-07-21

Published in
Proceedings of the 41st International Conference on Machine Learning (ICML) 2024
Total of pages

14

Subjects

Heteroscedastic regression

•

Covariance estimation

•

Taylor Induced Covariance

•

Task Agnostic Correlations

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
VITA  
Event nameEvent placeEvent date
41st International Conference on Machine Learning (ICML) 2024

Vienna, Austria

July 21-27, 2024

Available on Infoscience
May 19, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/208028
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés