Realand imaginary -time quantum state evolutions are crucial in physics and chemistry for exploring quantum dynamics, preparing ground states, and computing thermodynamic observables. On near -term devices, variational quantum time evolution is a promising candidate for these tasks, as the required circuit model can be tailored to the available devices' capabilities. Due to the evaluation of the quantum geometric tensor (QGT), however, this approach quickly becomes infeasible for relevant system sizes. Here, we propose a dual formulation for variational time evolution, which replaces the calculation of the QGT by solving a fidelity -based optimization to compute updates to the dynamics in each time step. We demonstrate our algorithm for the time evolution of the Heisenberg Hamiltonian and show that it accurately reproduces the system dynamics at a fraction of the cost of standard variational quantum time evolution algorithms.