The paramagnetic NMR study of HoM2N@C-80-I-h and Ho2MN@C-80-I-h nitride cluster fullerenes (M = Sc, Lu, Y) reveals strong dependence of Ho-induced paramagnetic shifts (delta(para)) in C-13 NMR spectra on the size of the diamagnetic metal in the cluster. In particular, the delta(para) value in HoY2N@C-80 is almost doubled in comparison to that in HoSc2N@C-80. X-ray magnetic circular dichroism studies show that all Ho-nitride cluster fullerenes have the same magnetic ground state of Ho3+. Point-charge ligand-field splitting calculations show that the increase of the M3+ radius in going from Sc to Y results in a considerable increase of the energy splitting between different J(z) states. This leads to a 19% higher magnetic anisotropy of Ho3+ in HoY2N@C-80 than in HoSc2N@C-80 at 300 K. Variations of the molecular geometry and cluster dynamics with the size of the cluster are found to have even greater influence on delta(para) values. This work shows that the magnetic properties of the species confined inside the fullerene cages can be tuned using the geometrical factors such as the cluster and the cage size.