Currently, more than half of the world’s population lives in cities, which leads to major changes in land use and land surface temperature (LST). The associated urban heat island (UHI) effects have multiple impacts on energy consumption and human health. A better understanding of how different land covers affect LST is necessary for mitigating adverse impacts, and supporting urban planning and public health management. This study explores a distance-based, a grid-based and a point-based analysis to investigate the influence of impervious surfaces, green area and waterbodies on LST, from large (distance and grid based analysis with 400 m grids) to smaller (point based analysis with 30 m grids) scale in the two mid-latitude cities of Paris and Geneva. The results at large scale confirm that the highest LST was observed in the city centers. A significantly positive correlation was observed between LST and impervious surface density. An anticorrelation between LST and green area density was observed in Paris. The spatial lag model was used to explore the spatial correlation among LST, NDBI, NDVI and MNDWI on a smaller scale. Inverse correlations between LST and NDVI and MNDWI, respectively, were observed. We conclude that waterbodies display the greatest mitigation on LST and UHI effects both on the large and smaller scale. Green areas play an important role in cooling effects on the smaller scale. An increase of evenly distributed green area and waterbodies in urban areas is suggested to lower LST and mitigate UHI effects.