In this paper, a nonlinear problem corresponding to a simplified Oldroyd-B model without convective terms is considered. Assuming the domain to be a convex polygon, existence of a solution is proved for small relaxation times. Continuous piecewise linear finite elements together with a Galerkin Least Square (GLS) method are studied for solving this problem. Existence and a priori error estimates are established using a Newton-chord fixed point theorem, a posteriori error estimates are also derived. An Elastic Viscous Split Stress (EVSS) scheme related to the GLS method is introduced. Numerical results confirm the theoretical predictions.