This article presents a mathematical framework based on information theory to compare multivariate sensory streams. Central to this approach is the notion of configuration: a set of distances between information sources, statistically evaluated for a given time span. As information distances capture simultaneously effects of physical closeness, intermodality, functional relationship and external couplings, a configuration can be interpreted as a signature for specific patterns of activity. This provides ways for comparing activity sequences by viewing them as points in an activity space. Results of experiments with an autonomous robot illustrate how this framework can be used to perform unsupervised activity classification.