We investigate redox levels in aqueous solution using a combination of ab initio molecular dynamics (MD) simulations and thermodynamic integration methods. The molecular dynamics are performed with both the semilocal Perdew-Burke-Ernzerhof functional and a nonlocal functional (rVV10) accounting for van der Waals (vdW) interactions. The band edges are determined through three different schemes, namely, from the energy of the highest occupied and of the lowest unoccupied Kohn-Sham states, from total-energy differences, and from a linear extrapolation of the density of states. It is shown that the latter does not depend on the system size while the former two are subject to significant finite-size effects. For the redox levels, we provide a formulation in analogy to the definition of charge transition levels for defects in crystalline materials. We consider the H+/H2 level defining the standard hydrogen electrode, the OH-/OH- level corresponding to the oxidation of the hydroxyl ion, and the H2O/OH- level for the dehydrogenation of water. In spite of the large structural modifications induced in liquid water, vdW interactions do not lead to any significant structural effect on the calculated band gap and band edges. The effect on the redox levels is also small since the solvation properties of ionic species are little affected by vdW interactions. Since the electronic properties are not significantly affected by the underlying structural properties, it is justified to perform hybrid functional calculations on the configurations of our MD simulations. The redox levels calculated as a function of the fraction α of Fock exchange are found to remain constant, reproducing a general behavior previously observed for charge transition levels of defects. Comparison with experimental values shows very good agreement. At variance, the band edges and the band gap evolve linearly with α. For α 0.40, we achieve a band gap, band-edge positions, and redox levels in overall good agreement with experiment.