Socioeconomic status (SES) plays a significant role in health and disease. At the same time, early-life conditions affect neural function and structure, suggesting the brain may be a conduit for the biological embedding of SES. Here, we investigate the brain anatomy signatures of SES in a large-scale population cohort aged 45–85 years. We assess both gray matter morphometry and tissue properties indicative of myelin content. Higher life course SES is associated with increased volume in several brain regions, including postcentral and temporal gyri, cuneus, and cerebellum. We observe more widespread volume differences and higher myelin content in the sensorimotor network but lower myelin content in the temporal lobe associated with childhood SES. Crucially, childhood SES differences persisted in adult brains even after controlling for adult SES, highlighting the unique contribution of early-life conditions to brain anatomy, independent of later changes in SES. These findings inform on the biological underpinnings of social inequality, particularly as they pertain to early-life conditions.