Human perception of a stimulus varies depending on the context in which the stimulus is presented. Such contextual modulation has often been explained by two basic neural mechanisms: lateral inhibition and spatial pooling. In the present study, we presented observers with a vernier stimulus flanked by single lines; observers' ability to discriminate the offset direction of the vernier stimulus deteriorated in accordance with both explanations. However, when the flanking lines were part of a geometric shape (i.e., a good Gestalt), this deterioration strongly diminished. These findings cannot be explained by lateral inhibition or spatial pooling. It seems that Gestalt factors play an important role in contextual modulation. We propose that contextual modulation can be used as a quantitative measure to investigate the rules governing the grouping of elements into meaningful wholes.