Random orientation of liquid-suspended magnetic nanoparticles (Ferrofluids) gives rise to a zero net magnetic orientation. An external magnetic field tends to align these nanoparticles into clusters, leading to a strong linear dichroism on a propagating wave. Using 10 nm-sized Fe3O4, we experimentally realize a polarization-sensitive magnetic modulator operating at terahertz wavelengths. We reached a modulation depth of 66% using a field as low as 35 mT. The proposed concept offers a solution towards fundamental terahertz magnetic modulators.