In this paper, we study high-resolution spectra of 19 stars that have metallicity estimates below -3.5 using at least two metallicity-sensitive photometric indices based on Pristine photometry. The purpose is to understand what kind of stars populate this parameter space, together with extremely metal-poor stars. This because we plan to extensively use the Pristine photometry to provide extremely metal-poor targets to the WEAVE spectroscopic survey and wish to understand the nature of possible contaminants. We find that this extreme sample of stars is heavily contaminated by variable stars, in particular short period eclipsing binaries. We thus found, serendipitously, eight double-lined spectroscopic eclipsing binaries that could be followed-up in future studies to provide reliable masses and distances for these systems. We also found two stars that have metallicity below -3.0, one of which may belong to the Gaia-Sausage-Enceladus structure. The lesson to be learned from this investigation is that to select truly metal-poor stars one should be able to remove all photometrically variable stars, which requires complementary information beyond the Pristine photometry. We show how the Gaia photometry can be used to remove about 85 per cent of the photometrically variable stars. Our investigation also shows that there is a clear potential for Pristine photometry to find double-lined spectroscopic binaries among short period eclipsing binaries.