We consider a scenario where local Lorentz invariance is violated by the existence of a preferred time direction at every space-time point. This scenario can arise in the context of quantum gravity and its description at low energies contains a unit time-like vector field which parameterizes the preferred direction. The particle physics tests of Lorentz invariance preclude a direct coupling of this vector to the fields of the Standard Model, but do not bear implications for dark matter. We discuss how the presence of this vector and its possible coupling to dark matter affect the evolution of the Universe. At the level of homogeneous cosmology the only effect of Lorentz invariance violation is a rescaling of the expansion rate. The physics is richer at the level of perturbations. We identify three effects crucial for observations: the rescaling of the matter contribution to the Poisson equation, the appearance of an extra contribution to the anisotropic stress and the scale-dependent enhancement of dark matter clustering. These effects result in distinctive features in the power spectra of the CMB and density fluctuations. Making use of the data from Planck and WiggleZ we obtain the most stringent cosmological constraints to date on departures from Lorentz symmetry. Our analysis provides the first direct bounds on deviations from Lorentz invariance in the dark matter sector.