We define a state space and a Markov process associated to the stochastic quantisation equation of Yang-Mills-Higgs (YMH) theories. The state space S is a nonlinear metric space of distributions, elements of which can be used as initial conditions for the (deterministic and stochastic) YMH flow with good continuity properties. Using gauge covariance of the deterministic YMH flow, we extend gauge equivalence ∼ to S and thus define a quotient space of "gauge orbits" O. We use the theory of regularity structures to prove local in time solutions to the renormalised stochastic YMH flow. Moreover, by leveraging symmetry arguments in the small noise limit, we show that there is a unique choice of renormalisation counterterms such that these solutions are gauge covariant in law. This allows us to define a canonical Markov process on O (up to a potential finite time blow-up) associated to the stochastic YMH flow.