Human perceptual development evolves in a stereotyped fashion, with initially limited perceptual capabilities maturing over the months or years following the commencement of sensory experience into robust proficiencies. This review focuses on the functional significance of these developmental progressions. Specifically, we review findings from studies of children who have experienced alterations of early development, as well as results from corresponding computational models, which have recently provided compelling evidence that specific attributes of early sensory experience are likely to be important prerequisites for later developing skills in several perceptual domains such as vision and audition. Notably, the limitations of early sensory experience have therein emerged as scaffolds, rather than hurdles, being causally responsible for the acquisition of later perceptual proficiencies, while dispensing with these limitations has the perhaps counter-intuitive consequence of compromising later development. These results have implications for understanding why normal trajectories of perceptual development are sequenced in the way that they are, help account for the perceptual deficits observed in individuals with atypical histories of sensory development, and serve as guidelines for the creation of more robust and effective training procedures for computational learning systems.