In perceptual learning, performance usually improves when observers train with one type of stimulus, for example, a bisection stimulus. Roving denotes the situation when, instead of one, two or more types of stimuli are presented randomly interleaved, for example, a bisection stimulus and a vernier. For some combinations of stimulus types, performance improves in roving situations whereas for others it does not. To investigate when roving impedes perceptual learning, we conducted four experiments. Performance improved, for example, when we roved a bisection stimulus and a vernier but not when we roved certain types of bisection stimuli. We propose that roving hinders perceptual learning when the stimulus types are clearly distinct from each other but still excite overlapping but not identical neural populations.