We have performed spectral interferometry with Rayleigh scattered light coming from a single speckle and emitted after resonant creation of excitons in GaAs quantum wells:: The experiment confirms the coherence of the secondary emission at early times, regardless of sample:parameters. but underlines the stochastic nature of a single speckle. The scattered electric field is calculated, correctly accounting for the center-of-mass quantization in the disordered potential. We find that the amplitudes of the electric fields emitted by different excitons are uncorrelated, so that the well-known Poisson distribution for speckle intensities applies.