Using silhouettes in uncontrolled environments typically requires handling occlusions as well as changing or cluttered backgrounds, which limits the applicability of most silhouette based methods. For the purpose of 3--D shape modeling, we show that representing generic 3--D surfaces as implicit surfaces lets us effectively address these issues. This desirable behavior is completely independent from the way the surface deformations are parametrized. To show this, we demonstrate our technique in three very different cases: Modeling the deformations of a piece of paper represented by an ordinary triangulated mesh; reconstruction and tracking a person's shoulders whose deformations are expressed in terms of Dirichlet Free Form Deformations; reconstructing the shape of a human face parametrized in terms of a Principal Component Analysis model.