Almost all optical diagnostics systems on ITER will be based on in-vessel metallic first mirrors. The possible deterioration of their surface reflectivity as a result of erosion by charge-exchange neutrals and re-deposition of material eroded from the plasma-facing components represents a serious concern for the reliability of spectroscopic and laser signals. A concerted effort within the tokamak community has been initiated to characterize these effects and seek mitigation methods. To date, the different damaging effects (erosion and deposition) have always been considered independently, neglecting any role that may be played by the substrate. In this contribution, we attempt to assess the influence of substrate material using different approaches: laboratory simulation experiments, mirror exposures in the divertor region of the TCV tokamak and numerical simulations using the Monte-Carlo code TRIDYN. Our results have potentially important implications for the choice of ITER first mirror materials.