Topographical disorientation refers to the selective inability to orient oneself in familiar surroundings. However, to date its neural correlates remain poorly understood. Here we use quantitative lesion analysis and a lesion network mapping approach in order to investigate seven patients with topographical disorientation. Our findings link not only the posterior parahippocampal gyrus (PHG) and retrosplenial cortex but also the lingual gyrus, the precuneus and the fusiform gyrus to topographical disorientation. We propose that topographical disorientation is due to the inability to integrate familiar landmarks within a framework of allocentric and egocentric orientation, supported by a neural network including the posterior PHG, the retrosplenial and the lingual cortex.