Elements in the neighborhood of a stimulus can modulate both the subjective perception of and the neural responses to this stimulus. Investigations of this contextual modulation usually focus on low level features such as the orientation difference between the target and its context. Recently, we introduced a paradigm in which contextual modulation cannot be explained by orientation differences between target and context per se. Instead, the overall structure of the context seemed to determine contextual modulation. Here, we show that edges of contextual gratings as well as isolated contextual lines are the main source of contextual suppression in this effect. Such suppressive contextual elements can be blocked by non-suppressive ones. We suggest that contextual elements reveal valences in loose analogy to chemical valences.