What is the `meaning' of a single spike? Spike-triggered averaging (`reverse correlations') yields the typical input just before a spike. Similarly, cross-correlations describe the probability of firing an output spike given (one additional) presynaptic input spike. In this paper we analytically calculate reverse and cross-correlations for a spiking neuron model with escape noise. The influence of neuronal parameters (such as the membrane time constant, the noise level, and the mean firing rate) on the form of the correlation function is illustrated. The calculation is done in the framework of a population theory that is reviewed. The relation of the population activity equations to population density methods is discussed. Finally we indicate the role of cross-correlations in spike-time dependent Hebbian plasticity.