We introduce a new concept of solution to the KPZ equation which is shown to extend the classical Cole-Hopf solution. This notion provides a factorisation of the Cole-Hopf solution map into a "universal" measurable map from the probability space into an explicitly described auxiliary metric space, composed with a new solution map that has very good continuity properties. The advantage of such a formulation is that it essentially provides a pathwise notion of a solution, together with a very detailed approximation theory. In particular, our construction completely bypasses the Cole-Hopf transform, thus laying the groundwork for proving that the KPZ equation describes the fluctuations of systems in the KPZ universality class.As a corollary of our construction, we obtain very detailed new regularity results about the solution, as well as its derivative with respect to the initial condition. Other byproducts of the proof include an explicit approximation to the stationary solution of the KPZ equation, a well-posedness result for the Fokker-Planck equation associated to a particle diffusing in a rough space-time dependent potential, and a new periodic homogenisation result for the heat equation with a space-time periodic potential. One ingredient in our construction is an example of a non-Gaussian rough path such that the area process of its natural approximations needs to be renormalised by a diverging term for the approximations to converge.