Automatic Skin Enhancement with Visible and Near-Infrared Image Fusion

Skin tones, portraits in particular, are of critical importance in photography and video, but a number of factors, such as pigmentation irregularities (e.g., moles, freckles), irritation, roughness, or wrinkles can reduce their appeal. Moreover, such “defects” are oftentimes enhanced by scene lighting conditions. Starting with the observations that melanin and hemoglobin, the key components of skin color, have little absorption in the near-infrared (NIR) part of the spectrum, and that the depth of light penetration in the epidermis is proportional to the incident light’s wavelength, we show that near-infrared images provide information that can be used to automatically smooth skin tones in a physically realistic manner. Specifically, we developed a prototype camera system that consists of capturing a pair of visible/near-infrared images and separating both of them into base and detail layers (akin to a low/high frequency decomposition) with the fast bilateral filter. Smooth and realistic output images are obtained by fusing the base layer of the visible image with the near-infrared detail layer. The proposed method delivers consistently good results across various skin types. The prototype system is currently in use at the Swiss Camera Museum in Vevey, Switzerland, where the visitors can take their pictures and e-mail themselves the results. In the process, we are collecting the users’ preference for either the “original” (visible) image or the “enhanced” (visible and NIR fused) image. The system has been deployed for three months. Preliminary statistics indicate that a large majority (79%) prefers the enhanced image.

Published in:
Proceedings of the 18th International Conference ACM MULTIMEDIA 2010
Presented at:
ACM MULTIMEDIA 2010, Florence, Italy, October 25-29, 2010
This paper and demo won the ACM Multimedia 2010 Demo Award. Note that the skin smoothing demo is also present in the Digital Revolution exhibit at the Swiss Camera Museum in Vevey, Switzerland, until December 2016. Visitors can take their own pictures and get them displayed before and after smoothing, and e-mail themselves the results.

Note: The status of this file is: Anyone

 Record created 2010-10-31, last modified 2020-07-30

Download fulltextPDF
External link:
Download fulltextURL
Rate this document:

Rate this document:
(Not yet reviewed)