Numerical simulations of dwarf galaxies have so far failed to reproduce the observed metallicity-luminosity relation, down to the regime of ultra-faint dwarfs (UFDs). We address this issue by exploring how the first generations of metal-free stars (Pop III) could help increase the mean metallicity ([Fe/H]) of those small and faint galaxies. We ran zoom-in chemo-dynamical simulations of 19 halos extracted from a Lambda Cold Dark Matter (CDM) cosmological box and followed their evolution down to redshift z=0. Models were validated not only on the basis of galaxy global properties, but also on the detailed investigation of the stellar abundance ratios ([alpha/Fe]). We identified the necessary conditions for the formation of the first stars in mini-halos and derived constraints on the metal ejection schemes. The impact of Pop III stars on the final metallicity of UFDs was evaluated by considering different stellar mass ranges for their initial mass function (IMF), the influence of pair-instability supernovae (PISNe), and their energetic feedback, as well as the metallicity threshold that marks the transition from the first massive stars to the formation of low-mass long-lived stars. The inclusion of Pop III stars with masses below 140 M-circle dot, and a standard IMF slope of -1.3 does increase the global metallicity of UFDs, although these are insufficient to resolve the tension with observations. The PISNe with progenitor masses above 140 M-circle dot do allow the metal content of UFDs to further increase. However, as PISNe are very rare and sometimes absent in the faintest UFDs, they have a limited impact on the global faint end of the metallicity-luminosity relation. Despite a limited number of spectroscopically confirmed members in UFDs, which make the stellar metallicity distribution of some UFDs uncertain, our analysis reveals that this is essentially the metal-rich tail that is missing in the models. The remaining challenges are thus both observational and numerical: (i) to extend high-resolution spectroscopy data samples and confirm the mean metallicity of the faintest UFDs; and (ii) to explain the presence of chemically enriched stars in galaxies with very short star formation histories.