Hold me tight! Influence of discriminative features on deep network boundaries

Important insights towards the explainability of neural networks reside in the characteristics of their decision boundaries. In this work, we borrow tools from the field of adversarial robustness, and propose a new perspective that relates dataset features to the distance of samples to the decision boundary. This enables us to carefully tweak the position of the training samples and measure the induced changes on the boundaries of CNNs trained on large-scale vision datasets. We use this framework to reveal some intriguing properties of CNNs. Specifically, we rigorously confirm that neural networks exhibit a high invariance to non-discriminative features, and show that very small perturbations of the training samples in certain directions can lead to sudden invariances in the orthogonal ones. This is precisely the mechanism that adversarial training uses to achieve robustness.

Published in:
[Advances in Neural Information Processing Systems 34]
Presented at:
Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020), [Virtual only], December 6-12, 2020

Note: The status of this file is: Anyone

 Record created 2020-09-29, last modified 2020-10-02

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)