In this work we discuss the Dynamically Orthogonal (DO) approximation of time dependent partial differential equations with random data. The approximate solution is expanded at each time instant on a time dependent orthonormal basis in the physical domain with fixed and small number of terms. Dynamic equations are written for the evolution of the basis as well as the evolution of the stochastic coefficients of the expansion. We analyze the case of a linear parabolic equation with random data and derive a theoretical bound for the approximation error of the S-terms DO solution by the corresponding S-terms best approximation, i.e. the truncated S-terms Karhunen-Loève expansion at each time instant, under the assumption that the latter is continuously differentiable in time. Properties of the DO approximations are analyzed on simple cases of deterministic equations with random initial data. Numerical tests are presented that confirm the theoretical bound and show potentials and limitations of the proposed approach.