This article is devoted to the numerical study of various finite-difference approximations to the stochastic Burgers equation. Of particular interest in the one-dimensional case is the situation where the driving noise is white both in space and in time. We demonstrate that in this case, different finite-difference schemes converge to different limiting processes as the mesh size tends to zero. A theoretical explanation of this phenomenon is given and we formulate a number of conjectures for more general classes of equations, supported by numerical evidence.