The spacing between target and flankers is one of the main factors of crowding. The closer the flankers are to the target, the stronger is crowding. Recently, it was proposed that crowding strength is determined by the distance between target and flanker centroids (Levi & Carney, 2009). Here, we determined vernier offset discrimination in the periphery with different flanker configurations. When the vernier was flanked by two vertical lines, thresholds increased. Thresholds decreased compared to the two-lines condition when each of the lines was complemented to form a rectangle. This is in line with the centroid hypothesis because the rectangles’ centroids are further away than the centroids of the single flankers. However, when crossing the upper and lower horizontal lines of the rectangles, performance deteriorated even though centroids are the same in this and the rectangle condition. These results can neither be explained by the spacing between the target and the flankers nor by centroid distance. Also simple pooling models fail to account for these results. We propose instead that grouping is a key factor in crowding: crowding decreases when target and flankers ungroup, crowding increases when target and flankers group.