This paper presents a novel method of enhancing image quality of face pictures using 3D and spectral information. Most conventional techniques directly work on the image data, shifting the skin color to a predefined skin tone, and so do not take into account the effects of shape and lighting. The proposed method first recovers the 3D shape of a face in an input image using a 3D morphable model. Then, using color constancy and inverse rendering techniques, specularities and the true skin color, i.e., its spectral reflectance, are recovered. The quality of the input image is improved by matching the skin reflectance to a predefined reference and reducing the amount of specularities. The method realizes the enhancement in a more physically accurate manner compared to previous ones. Subjective experiments on image quality demonstrates the validity of the proposed method.