Crowding, the deterioration of object recognition in clutter, is traditionally explained with models that are hierarchical, feedforward and local. These models suggest that a “bottleneck” at the earliest stages of visual processing leads to an irretrievable loss of information due to the pooling of target and flanker features, perhaps because of their combination within a single receptive field. A recent study used fMRI and population receptive field (pRF) mapping to estimate aggregate receptive field sizes in early visual areas under conditions of crowding (He et al., 2019). In area V2, pRF size was larger in a stronger crowding condition as compared to an easier one, suggesting that pRF sizes indicate the strength of the “bottleneck”. Here, we tested this assumption by using uncrowding, in which adding more flankers can lead to better performance. In accordance with local, feedforward models, pRF sizes in uncrowding should be the same or larger than in crowding. We estimated pRF sizes in three conditions: crowding, uncrowding and no crowding. We replicated previous results, showing that pRF size was increased in the crowding condition as compared to the no crowding condition. This was the case across visual areas V1 to V4. However, in the uncrowding condition, pRF size was significantly decreased, even compared to the no crowding condition. Again, this was true across all visual areas tested. Our findings not only show that there is “isolation” of the target from the flankers in the uncrowding condition – which may explain the higher task performance, but also provide evidence against purely feedforward models of crowding, including the “bottleneck” theory. We suggest that pRF size is modulated in a recurrent fashion, dependent on global context.