We propose a new formulation to the non-rigid structure-from-motion problem that only requires the deforming surface to meaning that its differential structure is preserved. This is a much weaker assumption than the traditional ones of isometry or conformality. We show that it is nevertheless sufficient to establish local correspondences between the surface in two different images and therefore to perform point-wise reconstruction using only up to first-order derivatives. We formulate differential constraints and solve them algebraically using the theory of resultants. We will demonstrate that our approach is more widely applicable, more stable in noisy and sparse imaging conditions and much faster than earlier ones, while delivering similar accuracy. The code is available at https//github.com/cvlab-epf1/diff-nrsfm/.