Aligning data distributions that underwent spectral distortions related to acquisition conditions is a key issue to improve the performance of classifiers applied to multi-temporal and multi-angular images. In this paper, we propose a feature extraction methodology, which aligns data manifolds based on their internal geometric structure and on a series of object correspondences highlighted on each image, or tie points. The weakly supervised manifold alignment (WeSMA) is a feature extractor that allows to define a common latent space, in which the images can be projected and processed by the same classifier. WeSMA relaxes the need for labeled pixels in all acquisitions of previous manifold alignment methods, an heavy constraint for remote sensing applications. Experiments on a set of World-View II images acquired at different viewing angles show the interest of the method that can compensate the spectral shift generated by the angular distortion without labels issued from the off-nadir image.