In this paper we find properties that are shared between two seemingly unrelated lossy source coding setups with side information. The first setup is when the source and side information are jointly Gaussian and the distortion measure is quadratic. The second setup is when the side information is an erased version of the source. We begin with the observation that in both these cases the Wyner-Ziv and conditional rate-distortion functions are equal. We further find that there is a continuum of optimal strategies for the conditional rate distortion problem in both these setups. Next, we consider the case when there are two decoders with access to different side-information sources. For the case when the encoder has access to the side information we establish bounds on the rate-distortion function and a sufficient condition for tightness. Under this condition, we find a characterization of the rate-distortion function for physically degraded side information. This characterization holds for both the Gaussian and erasure setups.