Sensory experience in early life shapes the mammalian brain. An impairment in the activity-dependent refinement of functional connections within developing visual cortex was identified here in a mouse model. Gene-targeted disruption of one isoform of glutamic acid decarboxylase prevented the competitive loss of responsiveness to an eye briefly deprived of vision, without affecting cooperative mechanisms of synapse modification in vitro. Selective, use-dependent enhancement of fast intracortical inhibitory transmission with benzodiazepines restored plasticity in vivo, rescuing the genetic defect. Specific networks of inhibitory interneurons intrinsic to visual cortex may detect perturbations in sensory input to drive experience-dependent plasticity during development.