In this work we discuss the dynamically orthogonal (DO) approximation of time dependent partial differential equations with random data. The approximate solution is expanded at each time instant on a time dependent orthonormal basis in the physical domain with a fixed and small number of terms. Dynamic equations are written for the evolution of the basis as well as the evolution of the stochastic coefficients of the expansion. We analyze the case of a linear parabolic equation with random data and derive a theoretical bound for the approximation error of the S-terms DO solution by the corresponding S-terms best approximation, i.e., the truncated S-terms Karhunen-Loeve expansion at each time instant. The bound is applicable on the largest time interval in which the best S-terms approximation is continuously time differentiable. Properties of the DO approximations are analyzed on simple cases of deterministic equations with random initial data. Numerical tests are presented that confirm the theoretical bound and show potentials and limitations of the proposed approach.