Crowding by neighboring elements leads to the deterioration of target discrimination. This phenomenon is traditionally explained with feedforward, hierarchical models. In these models, a “bottleneck” at the earliest stages of visual processing causes an irreversible loss of information, which is perhaps due to the pooling of target and flanker features when they fall within the same receptive field. He et al. (2019) used fMRI and showed that population receptive field (pRF) size in V2 is smaller in a weaker crowding condition as compared to a strong crowding condition. Their results are in line with the traditional explanation, with pRF size acting as a proxy for the strength of the “bottleneck”. Here, we estimated pRF sizes in three stimulus configurations, corresponding to no crowding, crowding and uncrowding, i.e., the alleviation of crowding due to the presence of additional flankers. Local, feedforward models suggest that pRF sizes in uncrowding should be the same as or larger than those in crowding. However, we found that, in fact, pRF sizes in the uncrowding condition are significantly smaller than in the other two conditions. This was true across early visual areas V1 to V4, with the exception of V3 in the no crowding vs. uncrowding comparison. We did not find any difference between the crowding and no crowding condition in any of the tested visual areas. Our results suggest that while in crowding, target and flanker features are combined because they fall within the same receptive field, in uncrowding, the target is isolated from the flankers to a greater extent through a decrease in pRF size. These findings contradict purely feedforward models of vision, rather suggesting that pRF size can be modulated through feedback modulation dependent on global context. We posit that recurrent processing plays a critical role in (un)crowding and vision in general.