In the periphery, discrimination of a target is usually impaired by flanking elements. This crowding effect is often explained in terms of pooling, ie target and flanker signals are averaged. The pooling hypothesis predicts stronger crowding when more flankers are presented. The centroid hypothesis, on the contrary, predicts that the more flankers are presented the weaker is the crowding. Here, we show that performance can both improve and deteriorate when more flankers are presented, depending on the layout of the flanker configuration. At 4° eccentricity, we determined offset discrimination thresholds for verniers embedded in regularly spaced arrays of flankers. The flankers were either shorter, of the same length, or longer than the vernier. We also presented the short flankers arranged in a spatially irregular fashion. Performance improved when increasing the number of shorter and longer regular flankers, and did not change for same-length, regular flankers. In contrast, performance deteriorated when increasing the number of irregular flankers. These results challenge both the pooling and centroid hypotheses. Instead, we propose that grouping determines crowding. Crowding is weak or absent when the vernier does not group with the flankers and strong when vernier and flankers group.