Mixed quantum-classical methods, such as surface hopping and Ehrenfest dynamics, have proven useful for describing molecular processes involving multiple electronic states. These methods require propagating many independent trajectories, which is computationally demanding. Therefore, we propose the single potential evaluation Ehrenfest dynamics (SPEED), a variation of Ehrenfest dynamics where all trajectories are propagated using a common local quadratic effective potential in the diabatic representation. This approach replaces the computational cost of propagating multiple trajectories with the evaluation of a single Hessian at each time step. We demonstrate the equivalence of standard Ehrenfest dynamics and SPEED in two realistic systems with (at most) quadratic diabatic potential energy surfaces and vibronic couplings: a quadratic vibronic coupling Hamiltonian model describing internal conversion in pyrazine and a model of atomic adsorption on a solid surface. The efficiency gain of our approach is particularly advantageous in on-the-fly ab initio applications. For this reason, we combined SPEED with the ALMO(MSDFT2) electronic structure method, which provides the diabatic potential describing charge transfer between two molecules. We find that SPEED qualitatively captures the temperature dependence of the hole transfer rate between two furan moieties and accurately predicts the final charge distribution after the collision. In contrast, but as expected, our approach is insufficient for describing photoisomerization of retinal due to the high anharmonicity of the potential energy surfaces already in the diabatic representation.