We address a simplified time-dependent parabolic problem that arises when modeling both viscoelastic fluids and elastic solids in a unified framework. A priori estimates are obtained for the continuous problem. Then implicit and semi-explicit (EVSS) time discretizations are considered. Stability estimates are obtained for the time-discretized problems. Numerical experiments show that appropriate convergence orders are obtained and that the implicit scheme is more accurate.