Within the ultimate goal of classifying universality in quantum many-body dynamics, understanding the relation between out-of-equilibrium and equilibrium criticality is a crucial objective. Models with power-law interactions exhibit rich well-understood critical behavior in equilibrium, but the out-of-equilibrium picture has remained incomplete, despite recent experimental progress. We construct the rich dynamical phase diagram of free-fermionic chains with power-law hopping and pairing and provide analytic and numerical evidence showing a direct connection between nonanalyticities of the return rate and zero crossings of the string order parameter. Our results may explain the experimental observation of so-called accidental dynamical vortices, which appear for quenches within the same topological phase of the Haldane model, as reported by Flaschner et al. [Nat. Phys. 14, 265 (2018)]. Our work is readily applicable to modern ultracold-atom experiments, not least because state-of-the-art quantum gas microscopes can now reliably measure the string order parameter, which, as we show, can serve as an indicator of dynamical criticality.