Quantum annealing is a contender to solve combinatorial optimization problems based on quantum dynamics. While significant efforts have been undertaken to investigate the quality of the solutions and the required run times, much less attention has been paid to understanding the dynamics of quantum annealing and the process leading to the solution during the sweep itself. In this comprehensive study, we investigate various aspects of the quantum annealing dynamics using different approaches. We perform quantum annealing, simulated quantum annealing, and classical annealing on several hundred instances of the Sherrington-Kirkpatrick model with intermediate system sizes up to 22 spins using numerical simulations. We observe qualitative differences between the quantum and classical methods, in particular at intermediate times, where a peak in the fidelity, also known as diabatic bump, appears for hard instances. Furthermore, we investigate the two-point correlation functions, which feature differences at intermediate times as well. At short times, however, the methods are similar again, which can be explained by relating the short-time expansion of quantum annealing to a high-temperature expansion, thus allowing one in principle to find the classical solution already at short times, albeit at prohibitive sampling cost.