TRIM28 is critical for the silencing of endogenous retroviruses (ERVs) in embryonic stem (ES) cells. Here, we reveal that an essential impact of this process is the protection of cellular gene expression in early embryos from perturbation by cis-acting activators contained within these retroelements. In TRIM28-depleted ES cells, repressive chromatin marks at ERVs are replaced by histone modifications typical of active enhancers, stimulating transcription of nearby cellular genes, notably those harboring bivalent promoters. Correspondingly, ERV-derived sequences can repress or enhance expression from an adjacent promoter in transgenic embryos depending on their TRIM28-sensitivity in ES cells. TRIM28-mediated control of ERVs is therefore crucial not just to prevent retrotransposition, but more broadly to safeguard the transcriptional dynamics of early embryos.