Learning Lightprobes for Mixed Reality Illumination

This paper presents the first photometric registration pipeline for Mixed Reality based on high quality illumination estimation by convolutional neural network (CNN) methods. For easy adaptation and deployment of the system, we train the CNN using purely synthetic images and apply them to real image data. To keep the pipeline accurate and efficient, we propose to fuse the light estimation results from multiple CNN instances, and we show an approach for caching estimates over time. For optimal performance, we furthermore explore multiple strategies for the CNN training. Experimental results show that the proposed method yields highly accurate estimates for photo-realistic augmentations.


Présenté à:
International Symposium on Mixed and Augmented Reality (ISMAR), Nantes, France, October 9–13, 2017
Année
2017
Laboratoires:


Note: Le statut de ce fichier est:


 Notice créée le 2017-07-10, modifiée le 2019-12-05

Preprint:
Télécharger le document
PDF

Évaluer ce document:

Rate this document:
1
2
3
 
(Pas encore évalué)