000217899 001__ 217899
000217899 005__ 20181007231405.0
000217899 0247_ $$2doi$$a10.5075/epfl-thesis-6891
000217899 02470 $$2urn$$aurn:nbn:ch:bel-epfl-thesis6891-7
000217899 02471 $$2nebis$$a10616842
000217899 037__ $$aTHESIS_LIB
000217899 041__ $$aeng
000217899 088__ $$a6891
000217899 245__ $$aTemplate-based Monocular 3-D Shape Reconstruction And Tracking Using Laplacian Meshes
000217899 269__ $$a2016
000217899 260__ $$aLausanne$$bEPFL$$c2016
000217899 300__ $$a172
000217899 336__ $$aTheses
000217899 502__ $$aProf. Mark Pauly (président) ; Prof. Pascal Fua (directeur de thèse) ; Prof. Wenzel Jakob, Prof. Adrien Bartoli, Prof. Lloyd Smith (rapporteurs)
000217899 520__ $$aThis thesis addresses the problem of recovering the 3-D shape of a deformable object in single images, or image sequences acquired by a monocular video camera, given that a 3-D template shape and a template image of the object are available. While being a very challenging problem in computer vision, being able to reconstruct and track 3-D deformable objects in videos allows us to develop many potential applications ranging from sports and entertainments to engineering and medical imaging. This thesis extends the scope of deformable object modeling to real-world applications of fully 3-D modeling of deformable objects from video streams with a number of contributions.     We show that by extending the Laplacian formalism, which was first introduced in the Graphics community to regularize 3-D meshes, we can turn the monocular 3-D shape reconstruction of a deformable object given correspondences with a reference image into a much better-posed problem with far fewer degrees of freedom than the original one. This has proved key to achieving real-time performance while preserving both sufficient flexibility and robustness. Our real-time 3-D reconstruction and tracking system of deformable objects can very quickly reject outlier correspondences and accurately reconstruct the object shape in 3D. Frame-to-frame tracking is exploited to track the object under difficult settings such as large deformations, occlusions, illumination changes, and motion blur.     We present an approach to solving the problem of dense image registration and 3-D shape reconstruction of deformable objects in the presence of occlusions and minimal texture. A main ingredient is the pixel-wise relevancy score that we use to weigh the influence of the image information from a pixel in the image energy cost function. A careful design of the framework is essential for obtaining state-of-the-art results in recovering 3-D deformations of both well- and poorly-textured objects in the presence of occlusions.     We study the problem of reconstructing 3-D deformable objects interacting with rigid ones. Imposing real physical constraints allows us to model the interactions of objects in the real world more accurately and more realistically. In particular, we study the problem of a ball colliding with a bat observed by high speed cameras. We provide quantitative measurements of the impact that are compared with simulation-based methods to evaluate which simulation predictions most accurately describe a physical quantity of interest and to improve the models.     Based on the diffuse property of the tracked deformable object, we propose a method to estimate the environment irradiance map represented by a set of low frequency spherical harmonics. The obtained irradiance map can be used to realistically illuminate 2-D and 3-D virtual contents in the context of augmented reality on deformable objects. The results compare favorably with baseline methods.     In collaboration with Disney Research, we develop an augmented reality coloring book application that runs in real-time on mobile devices. The app allows the children to see the coloring work by showing animated characters with texture lifted from their colors on the drawing. Deformations of the book page are explicitly modeled by our 3-D tracking and reconstruction method. As a result, accurate color information is extracted to synthesize the character's texture.
000217899 6531_ $$adeformable surfaces
000217899 6531_ $$amonocular shape recovery
000217899 6531_ $$aLaplacian formalism
000217899 6531_ $$aaugmented reality
000217899 6531_ $$atemplate matching
000217899 6531_ $$aimage registration
000217899 6531_ $$aocclusions
000217899 6531_ $$aminimal texture
000217899 6531_ $$aspherical harmonics
000217899 6531_ $$ainteractive books
000217899 700__ $$0246262$$aNgo, Tien Dat$$g211303
000217899 720_2 $$0240252$$aFua, Pascal$$edir.$$g112366
000217899 8564_ $$s52002623$$uhttps://infoscience.epfl.ch/record/217899/files/EPFL_TH6891.pdf$$yn/a$$zn/a
000217899 909C0 $$0252087$$pCVLAB$$xU10659
000217899 909CO $$ooai:infoscience.tind.io:217899$$pDOI$$pIC$$pthesis$$qDOI2$$qthesis-bn2018
000217899 917Z8 $$x108898
000217899 917Z8 $$x108898
000217899 917Z8 $$x108898
000217899 918__ $$aIC$$cISIM$$dEDIC
000217899 919__ $$aCVLAB
000217899 920__ $$a2016-3-23$$b2016
000217899 970__ $$a6891/THESES
000217899 973__ $$aEPFL$$sPUBLISHED
000217899 980__ $$aTHESIS