Contextual signals might supervise discovery of coherently varying information between cortical modules computing different functions of their receptive field input. This hypothesis is explored in two sets of computational experiments, one studying the effects on learning of long-range unidirectional contextual signals mediated by intervening processors, and the other showing contextually supervised discovery of a high-order variable in a multi-layer network.