Scattering between charges and collective modes in materials governs phenomena such as electrical resistance, energy dissipation, and phase switching. Studying such scattering requires simultaneous access to ultrafast and momentum-resolved dynamics of single-particle and collective excitations, which remains an experimental challenge. Here, we present time- and momentum-resolved electron energy loss spectroscopy, and we apply it to graphite, demonstrating that large (Δq ≃1.2 Å−1) photoexcited electron-hole pockets induce a renormalization of in-plane and bulk plasmons. This effect is explained by intra- and intervalley scattering processes mediated by E2g and A′1 phonon modes, which we directly observe via ultrafast electron diffraction and identify via ab initio calculations. Conversely, smaller electron-hole pockets (Δq ≃0.7 Å−1) result in the renormalization of in-plane plasmons, which can only be partially explained by phonon-mediated scattering and thermal expansion. Our results highlight the importance of combining momentum- and time-resolved information to elucidate electronic scattering processes.