Using the cosmological simulations IllustrisTNG, we perform a comprehensive analysis of quiescent, massive galaxies at z ≳ 3. The goal is to understand what suppresses their star formation so early in cosmic time, and how other similar mass galaxies remain highly star forming. As a first-order result, the simulations are able to produce massive, quiescent galaxies in this high-redshift regime. We find that active galactic nucleus (AGN) feedback is the primary cause of halting star formation in early, massive galaxies. Not only do the central, supermassive black holes (SMBHs) of the quenched galaxies have earlier seed times, but they also grow faster than in star-forming galaxies. As a result, the quenched galaxies are exposed to AGN feedback for longer, and experience the kinetic, jet mode of the AGN feedback earlier than the star-forming galaxies. The release of kinetic energy reduces inflows of gas while likely maintaining outflows, which keeps a low cold gas fraction and decreases the star formation of the galaxies down to a state of quiescence. In addition to AGN feedback, we also investigate the influence of the large-scale environment. While mergers do not play a significant role in the quenching process, the quenched galaxies tend to reside in more massive haloes and denser regions during their evolution. As this provides a greater initial amount of infalling gas to the galaxies, the large-scale environment can mildly affect the fate of the central SMBH growth and, via AGN feedback, contribute to star formation quenching.