Memory is a process that works through the identification of social references within a spatial framework, without physical marks necessarily found in the tangible present. Nonetheless, these references subsist in representations from the past and continue to influence current urban realities. A city is an evolving structure containing different artifacts that relate directly to their importance to a given social group. Usually, but not exclusively, such places have strong historical and memorial significance and are recognized as heritage. Thus, as spaces evolve, inhabitants’ emotional bonds are sometimes at odds with the interactions of other actors. These dual interactions generate rivalries and controversies about uses and perceptions of the space. Traces left on the collective memory by violent urbanization processes can be identified by understanding the city through such intangible experiences. Using these principles—notably that forgetting is not equivalent to the destruction of a mnemonic device—we analyze the spatial and social experiences of controversial memories in two different cultural contexts; the Historic Center of Mexico City (Mexico) and the Gulou area of Beijing (China). The Mexican case study looks at a dominant discourse of urban heritage focused on tangible heritage conservation, which has often ignored the importance of local social memories. The Chinese case study scrutinizes the community’s memories in the context of rapid and radical transformation led by the local government’s urban policies and regulations.