This study presents a new algorithm developed in order to remove instabilities observed in the simulation of unsteady viscoelastic fluid flows in the framework of the spectral element method. In this study, we consider a particular model of the finite extensible nonlinear elastic family, FENE-P, but the method could be applied to other differential constitutive equations. Two distinct constraints for the FENE-P equation are imposed: (i) the square of the corresponding finite extensibility parameter of the polymer must be an upper limit for the trace of the conformation tensor and (ii) the eigenvalues of the conformation tensor should remain positive at all steps of the simulation. Negative eigenvalues cause the unbounded growth of instabilities in the flow. The proposed transformation is an extension of the matrix logarithm formulation originally presented by Fattal and Kupferman [1,2]. To evaluate the capability of this new algorithm with the classical conformation tensor, comprehensive studies have been done based on the linear stability analysis to show the influence of this method on the resulting eigenvalue spectra and explain its success to tackle high Weissenberg numbers. With this new method one can tackle high Weissenberg number flow at values of practical interest. A neat improvement of the computational algorithm with stable convergence has been demonstrated in this study. (C) 2010 Elsevier Ltd. All rights reserved.