Stress often affects our social lives. When undergoing high-level or persistent stress, individuals frequently retract from social interactions and become irritable and hostile. Predisposition to antisocial behaviours — including social detachment and violence — is also modulated by early life adversity; however, the effects of early life stress depend on the timing of exposure and genetic factors. Research in animals and humans has revealed some of the structural, functional and molecular changes in the brain that underlie the effects of stress on social behaviour. Findings in this emerging field will have implications both for the clinic and for society.