Machine-learning models have emerged as a very effective strategy to sidestep time-consuming electronic-structure calculations, enabling accurate simulations of greater size, time scale, and complexity. Given the interpolative nature of these models, the reliability of predictions depends on the position in phase space, and it is crucial to obtain an estimate of the error that derives from the finite number of reference structures included during model training. When using a machine-learning potential to sample a finite-temperature ensemble, the uncertainty on individual configurations translates into an error on thermodynamic averages and leads to a loss of accuracy when the simulation enters a previously unexplored region. Here, we discuss how uncertainty quantification can be used, together with a baseline energy model, or a more robust but less accurate interatomic potential, to obtain more resilient simulations and to support active-learning strategies. Furthermore, we introduce an on-the-fly reweighing scheme that makes it possible to estimate the uncertainty in thermodynamic averages extracted from long trajectories. We present examples covering different types of structural and thermodynamic properties and systems as diverse as water and liquid gallium.