We study the evolution of spectral early-type galaxies in clusters, groups, and the field up to redshift 0.9 using the ESO Distant Cluster Survey (EDisCS) dataset. We measure structural parameters (circularized half-luminosity radii R-e, surface brightness I-e, and velocity dispersions sigma) for 154 cluster and 68 field galaxies. On average, we achieve precisions of 10% in R-e, 0.1 dex in log I-e, and 10% in s. We sample approximate to 20% of cluster and approximate to 10% of field spectral early-type galaxies to an I band magnitude in a 1 arcsec radius aperture as faint as I-1 = 22. We study the evolution of the zero point of the fundamental plane (FP) and confirm results in the literature, but now also for the low cluster velocity dispersion regime. Taken at face value, the mass-to-light ratio varies as Delta log M/L-B = (-0.54 +/- 0.01)z = (-1.61 +/- 0.01) log(1 + z) in clusters, independent of their velocity dispersion. The evolution is stronger (Delta log M/L-B = (-0.76 +/- 0.01)z = (-2.27 +/- 0.03) log(1 + z)) for field galaxies. A somewhat milder evolution is derived if a correction for incompleteness is applied. A rotation in the FP with redshift is detected with low statistical significance. The alpha and beta FP coefficients decrease with redshift, or, equivalently, the FP residuals correlate with galaxy mass and become progressively negative at low masses. The effect is visible at z >= 0.7 for cluster galaxies and at lower redshifts z >= 0.5 for field galaxies. We investigate the size evolution of our galaxy sample. In agreement with previous results, we find that the half-luminosity radius for a galaxy with a dynamical or stellar mass of 2 x 10(11) M-circle dot varies as (1 + z)(-1.0+/-0.3) for both cluster and field galaxies. At the same time, stellar velocity dispersions grow with redshift, as (1 + z)(0.59+/-0.10) at constant dynamical mass, and as (1 + z)(0.34+/-0.14) at constant stellar mass. The measured size evolution reduces to R-e proportional to (1 + z)(-0.5+/-0.2) and sigma proportional to (1 + z)(0.41+/-0.08), at fixed dynamical masses, and R-e proportional to (1 + z)(-0.68+/-0.4) and sigma proportional to (1 + z)(0.19+/-0.10), at fixed stellar masses, when the progenitor bias (PB, galaxies that locally are of spectroscopic early-type, but are not very old, disappear progressively from the EDisCS high-redshift sample; often these galaxies happen to be large in size) is taken into account. Taken together, the variations in size and velocity dispersion imply that the luminosity evolution with redshift derived from the zero point of the FP is somewhat milder than that derived without taking these variations into account. When considering dynamical masses, the effects of size and velocity dispersion variations almost cancel out. For stellar masses, the luminosity evolution is reduced to L-B proportional to (1+ z)(1.0) for cluster galaxies and L-B proportional to (1+ z)(1.67) for field galaxies. Using simple stellar population models to translate the observed luminosity evolution into a formation age, we find that massive (> 10(11) M-circle dot) cluster galaxies are old (with a formation redshift z(f) > 1.5) and lower mass galaxies are 3-4 Gyr younger, in agreement with previous EDisCS results from color and line index analyses. This confirms the picture of a progressive build-up of the red sequence in clusters with time.