The thermal boundary conductance (TBC) of metal/dielectric couples was measured for a large variety of metals on silicon, sapphire, and diamond using time domain thermoreflectance and compared to data previously obtained on diamond. In the case of silicon, HF-cleaned and RF-etched surfaces were tested. The detailed structure of these interfaces was studied, allowing distinction of two different cases of M/Si couples: (i) some amount of interfacial reaction exists for both surface terminations, resulting in similar TBCs; and (ii) chemically abrupt interfaces are achieved, resulting in TBC values that are always lower for RF-etched samples. The TBC values obtained on different substrates allowed identifying a tendency of the TBC to scale with the maximum transmission limit. A possible influence of the substrate was evaluated using both the diffuse mismatch model (DMM), which predicts a strong dependence on the substrate properties, and a newly developed approach based on the metal irradiance (IM), which predicts no dependence on the substrate properties. The DMM was implemented using a Debye model with either a linear (DMMLinear) or a Sine-Type (DMMSineType) dispersion, while the IM was implemented using a Sine-Type (IMSineType) dispersion. The DMMLinear and the IMSineType were found to be more suitable than the DMMSineType and to be equally precise in predicting TBC at metal/silicon and metal/sapphire interfaces. The IMSineType is found to be better suited than both the DMMLinear and the DMMSineType to predict TBC at metal/diamond interfaces. IMSineType being the only model tested that is suitable for all three substrates, it appears to be the most appropriate choice. As a corollary, we find that the TBC dependence on substrate properties is much weaker than predicted by the DMM.