Current commercial outdoor Mobile AR applications rely mostly on GPS antennas, digital compasses and accelerometers. Due to imprecise readings, the 2D placement of points of interest (POI) on the display can be uncorrelated with reality. We present a novel method to geo-locate a mobile device by rec- ognizing what is captured by its camera. A visual recognition algo- rithm in the cloud is used to identify geo-located reference images that match the camera’s view. Upon correct identification, fusion of sensor data from the mobile device and the server is used to es- tablish the most probable location. The corrected geo-location can then be used on the mobile device for further POI representation or as a better initialization step for further continuos tracking.