Maturation of the auditory system in early childhood significantly influences the development of language-related perceptual and cognitive abilities. This study aims to provide insights into the neurophysiological changes underlying auditory processing and speech-sound discrimination in the first two years of life. We conducted a study using high-density electroencephalography (EEG) to longitudinally record cortical auditory event-related potentials (CAEP) in response to synthesized syllable sounds with pitch/duration change in a cohort of 79 extremely and very preterm-born infants without developmental disorders. EEG were recorded at 6 timepoints from term to 24 months corrected age, using a pseudorandom oddball paradigm. We found that the infant-P1 component of CAEP showed decreasing latency with age and more focalized cortical source stabilizing in the left primary auditory cortex by 6 months. By 6 months, a negative infant-N1 component emerged, its amplitude increasing with age and source localization showing increasing distribution over the left temporal, parietal and frontal lobes. Mismatch responses demonstrated significant differences in auditory discrimination capabilities starting from 6 months, indicating the infants' ability to detect phonetic differences. There was no correlation between infant-P1 latency, infant-P1 amplitude or mismatch response at term age and gestational age. This study suggests that cortical sound detection occurs very early and is not significantly influenced by the extent of prematurity but rather by corrected age. Early sound detection is followed by cortical sound content processing from about 6 months, with gradual organization along the cortical auditory dorsal stream and mirror neuron system in the first two years of life. Auditory discrimination of speech sounds also significantly changes from around 6 months of age.