Quantum mechanical simulations that include the effects of the liquid environment are highly relevant for the characterization of solid-liquid interfaces, which is crucial for the design of a wide range of devices. In this work we present a rigorous and systematic study of the band alignment of semiconductors in aqueous solutions by contrasting a range of hybrid explicit/implicit models against explicit atomistic simulations based on density-functional theory. We find that consistent results are obtained provided that the first solvation shell is treated explicitly. Interestingly, the first molecular layer of explicit water is only relevant for the pristine surfaces without dissociatively adsorbed water, hinting at the importance of saturating the surface with quantum mechanical bonds. By referencing the averaged electrostatic potentials of explicit and implicit water against vacuum, we provide absolute alignments, finding maximal differences of only similar to 0.1-0.2 V. Furthermore, the implicit reference potential is shown to exhibit an intrinsic offset of -033 V with respect to vacuum, which is traced back to the absence of an explicit water surface in the implicit model. These results pave the way for accurate simulations of solid-liquid interfaces using minimalistic explicit/implicit models.