Interpolation of data on non-Euclidean spaces is an active research area fostered by its numerous applications. This work considers the Hermite interpolation problem: finding a sufficiently smooth manifold curve that interpolates a collection of data points on a Riemannian manifold while matching a prescribed derivative at each point. A novel procedure relying on the general concept of retractions is proposed to solve this problem on a large class of manifolds, including those for which computing the Riemannian exponential or logarithmic maps is not straightforward, such as the manifold of fixed-rank matrices. The well-posedness of the method is analyzed by introducing and showing the existence of retraction-convex sets, a generalization of geodesically convex sets. A classical result on the asymptotic interpolation error of Hermite interpolation is extended to the manifold setting. Finally numerical experiments on the manifold of fixed-rank matrices and the Stiefel manifold of matrices with orthonormal columns illustrate these results and the effectiveness of the method.