The visual representation of the world is often assumed to be retinotopic, and many visual brain areas are indeed organized retinotopically. Visual perception, however, is not based on a reference frame anchored in retinotopic coordinates. For example, when an object moves, motion of its constituent parts is perceived relative to the object rather than in retinotopic coordinates. The moving object thus serves as a nonretinotopic reference system for computing the properties of its parts. It is largely unknown how the brain accomplishes this feat. Here, we used the Ternus-Pikler display to pit retinotopic processing in a stationary reference system against nonretinotopic processing in a moving one. Using 7T fMRI, we found that the average blood-oxygen-level dependent activations in V1, V2, and V3 reflected the retinotopic properties, but not the nonretinotopic percepts, of the Ternus-Pikler display. In the human motion processing complex (hMT+), activations were compatible with both retinotopic and nonretinotopic encoding. Thus, hMT+ may be the first visual area encoding the nonretinotopic percepts of the Ternus-Pikler display.