The diffuse stellar component of galaxy clusters made up of intergalactic stars is termed the intracluster light (ICL). Although there is a developing understanding of the mechanisms by which the ICL is formed, no strong consensus has yet been reached on which objects the stars of the ICL are primarily sourced from. We investigate the assembly of the ICL starting approximately 10 Gyr before z = 0 in 11 galaxy clusters (halo masses between ∼1 ×10 14 and ∼7 ×10 14 M⊙at z ≈0) in the HORIZON-AGN simulation. By tracking the stars of galaxies that fall into these clusters past cluster infall, we are able to link almost all of the z ≈0 ICL back to progenitor objects. Satellite stripping, mergers, and pre-processing are all found to make significant contributions to the ICL, but any contribution from in situ star formation directly into the ICL appears ne gligible. Ev en after compensating for resolution effects, we find that approximately 90 per cent of the stacked ICL of the 11 clusters that is not pre-processed should come from galaxies infalling with stellar masses abo v e 10 9 M⊙, with roughly half coming from infalling galaxies with stellar masses within half a dex of 10 11 M⊙. The fact that the ICL appears largely sourced from such massive objects suggests that the ICL assembly of any individual cluster may be principally stochastic.