Current advances in ultrafast electron microscopy make it possible to combine optical pumping of a nanostructure and electron beam probing with sub-Angstrom and femtosecond spatiotemporal resolution. We present a theory predicting that this technique can reveal a rich out-of-equilibrium dynamics of plasmon excitations in graphene and graphite samples. In a disruptive departure from the traditional probing of nanoscale excitations based on the identification of spectral features in the transmitted electrons, we show that the measurement of angle-resolved, energy-integrated inelastic electron scattering can trace the temporal evolution of plasmons in these structures and provide momentum-resolved mode identification, thus avoiding the need for highly monochromatic electron beams and the use of electron spectrometers. This previously unexplored approach to study the ultrafast dynamics of optical excitations can be of interest to understand and manipulate polaritons in 2D semiconductors and other materials exhibiting a strong thermo-optical response.