We consider robust feedback control of time-varying, linear discrete-time systems operating over a finite horizon. For such systems, we consider the problem of designing robust causal controllers that minimize the expected value of a convex quadratic cost function, subject to mixed linear state and input constraints. Determination of an optimal control policy for such problems is generally computationally intractable, but suboptimal policies can be computed by restricting the class of admissible policies to be affine on the observation. By using a suitable re-parameterization and robust optimization techniques, these approximations can be solved efficiently as convex optimization problems. We investigate the loss of optimality due to the use of such affine policies. Using duality arguments and by imposing an affine structure on the dual variables, we provide an efficient method to estimate a lower bound on the value of the optimal cost function for any causal policy, by solving a cone program whose size is a polynomial function of the problem data. This lower bound can then be used to quantify the loss of optimality incurred by the affine policy.