The structure of eukaryotic chromatin directly influences gene function, and is regulated by chemical modifications of the core histone proteins. Modification of the human histone H4 N-terminal tail region by the small ubiquitin-like modifier protein, SUMO-3, is associated with transcription repression. However, the direct effect of sumoylation on chromatin structure and function remains unknown. Therefore, we employed a disulfide-directed strategy to generate H4 homogenously and site-specifically sumoylated at Lys-12 (suH4ss). Chromatin compaction and oligomerization assays with nucleosomal arrays containing suH4ss established that SUMO-3 inhibits array folding and higher order oligomerization, which underlie chromatin fiber formation. Moreover, the effect of sumoylation differed from that of acetylation, and could be recapitulated with the structurally similar protein ubiquitin. Mechanistic studies at the level of single nucleosomes revealed that, unlike acetylation, the effect of SUMO-3 arises from the attenuation of long-range internucleosomal interactions more than from the destabilization of a compacted dinucleosome state. Altogether, our results present the first insight on the direct structural effects of histone H4 sumoylation and reveal a novel mechanism by which SUMO-3 inhibits chromatin compaction.