Crowding is traditionally thought to occur by local interactions between the target and the neighboring flankers, for example, by pooling neural responses corresponding to both the target and flankers. Accordingly, crowding is thought to occur within a small region around the target (Bouma’s window). Most of these explanations fit well in the classic hierarchical, feedforward framework of vision, where spatially confined neural circuits determine perception. We will show that crowding of a Vernier depends on the spatial layout of the entire stimulus configuration across large parts of the visual field. Obviously, low level vision cannot be explained by classic models of vision and crowding. We propose that grouping is key in crowding. Only when the Vernier groups with the flankers is crowding strong. When the Vernier stands out from the configuration, crowding is weak. Hence, one needs to understand how humans perceive the overall configuration to understand low level crowding. We will discuss the philosophical implications of our approach and present a large scale neural model, where neurons communicate across the entire visual field to compute grouping.