In this letter, we prove that the dynamical model of a quadrotor subject to linear rotor drag effects is differentially flat in its position and heading. We use this property to compute feedforward control terms directly from a reference trajectory to be tracked. The obtained feedforward terms are then used in a cascaded, nonlinear feedback control law that enables accurate agile flight with quadrotors. Compared to the state-of-the-art control methods, which treat the rotor drag as an unknown disturbance, our method reduces the trajectory tracking error significantly. Finally, we present a method based on a gradient-free optimization to identify the rotor drag coefficients, which are required to compute the feedforward control terms. The new theoretical results are thoroughly validated trough extensive comparative experiments.