In crowding, perception of a target deteriorates in the presence of nearby flankers. In the traditional feedforward framework of vision, only elements within Bouma’s window interfere with the target and adding more elements always leads to stronger crowding. Crowding is usually studied with sparse displays that involve only a few flankers, as too many flankers lead to a combinatorial explosion of display configurations. To deal with these challenges, Van der Burg et al. (2017) proposed a paradigm to measure crowding in dense displays using genetic algorithms. In their study, displays were selected and combined over several generations to maximize human performance. Van der Burg et al. found that only the target’s nearest neighbours affect performance in their displays. Here, we used the same paradigm, but the displays were selected according to the performance of visual crowding models. We found that all models based on the traditional framework of vision tested so far produce results in which all elements within Bouma’s window affect performance in dense displays, contrary to human behavior. The only model that explains the results of Van der Burg et al. has a dedicated grouping process. We conclude that a grouping stage is crucial to understand visual processing.