We present a systematic study of the phase diagram of LiHoxY1-xF4 (0.25 <= x <= 1) Ising ferromagnets obtained from neutron scattering measurements and mean-field calculations. We show that while the thermal phase transition decreases linearly with dilution, as predicted by mean-field theory, the critical transverse field at the quantum critical point is suppressed much faster. This behavior is related to competition between off-diagonal dipolar coupling and quantum fluctuations that are tuned by doping and applied field, respectively. In this paper, we quantify the deviation of the experimental results from mean-field predictions, with the aim that this analysis can be used in future theoretical efforts towards a quantitative description.