Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression
 
conference paper

TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression

Shukla, Megh  
•
Salzmann, Mathieu  
•
Alahi, Alexandre  
July 21, 2024
Proceedings of the 41st International Conference on Machine Learning (ICML) 2024
41st International Conference on Machine Learning (ICML) 2024

Deep heteroscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. However, recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. While the literature addresses this by proposing alternate formulations to mitigate the impact of the predicted covariance, we focus on improving this predicted covariance itself. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? (2) In the absence of supervision, how can we quantify the accuracy of covariance estimation? We address (1) with a Taylor Induced Covariance (TIC), which captures the randomness of the predicted mean by incorporating its gradient and curvature through the second order Taylor polynomial. Furthermore, we tackle (2) by introducing the Task Agnostic Correlations (TAC) metric, which combines the notion of correlations and absolute error to evaluate the covariance. We evaluate TIC-TAC across multiple experiments spanning synthetic and real-world datasets. Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of negative log-likelihood. We make our code available at: https://github.com/vita-epfl/TIC-TAC

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

CameraReady_ICML_2024__TIC_TAC.pdf

Type

Postprint

Version

http://purl.org/coar/version/c_ab4af688f83e57aa

Access type

openaccess

License Condition

CC BY-NC-ND

Size

5.58 MB

Format

Adobe PDF

Checksum (MD5)

d72ed64f60242a4383f15af4a1a90d7e

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés