We present a Residual Logit (ResLogit) model for seamlessly integrating a data-driven Deep NeuralNetwork (DNN) architecture in the random utility maximization paradigm. DNN models suchas the Multi-layer Perceptron (MLP) have shown remarkable success in modelling complex dataaccurately, but recent studies have consistently demonstrated that their black-box properties areincompatible with discrete choice analysis for the purpose of interpreting decision making behaviour.Our proposed machine learning choice model is a departure from the conventional feed-forward MLPframework by using a dynamic residual neural network learning based approach. Our proposedmethod can be formulated as a Generalized Extreme Value (GEV) random utility maximizationmodel for greater flexibility in capturing unobserved heterogeneity. It can generate choice modelstructures where the covariance between random utilities is estimated and incorporated into therandom error terms, allowing for a richer set of higher-order substitution patterns than a standardlogit might be able to achieve. We describe the process of our model estimation and examine therelative empirical performance and econometric implications on two mode choice experiments. Weanalyzed the behavioural and theoretical properties of our methodology. We showed how modelinterpretability is possible, while also capturing the underlying complex and unobserved behaviouralheterogeneity effects in the residual covariance matrices.