Deep convolutional neural networks have recently emerged as a state-of-the art tool in detection of seizures. Such models offer the ability to extract complex nonlinear representations of an electroencephalogram (EEG) signal which can improve accuracy over methods relying on hand-crafted features. However, neural networks are susceptible to confounding artifacts commonly present in EEG signals and are notoriously difficult to interpret. In this work, we present a neural-network based algorithm for seizure detection which leverages recent advances in information theory to construct a signal representation containing the minimal amount of information necessary to discriminate between seizure and normal brain activity. We show our approach automatically learns representations that ignore common signal artifacts and which encode medically relevant information from the raw signal.