Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks

In this paper, we present a novel method for real-time 3D hand pose estimation from single depth images using 3D Convolutional Neural Networks (CNNs). Image-based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN-based method, taking a 3D volumetric representation of the hand depth image as input and extracting 3D features from the volumetric input, can capture the 3D spatial structure of the hand and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. To further improve the estimation accuracy, we propose applying the 3D deep network architectures and leveraging the complete hand surface as intermediate supervision for learning 3D hand pose from depth images. Extensive experiments on three challenging datasets demonstrate that our proposed approach outperforms baselines and state-of-the-art methods. A cross-dataset experiment also shows that our method has good generalization ability. Furthermore, our method is fast as our implementation runs at over 91 frames per second on a standard computer with a single GPU.


Published in:
Ieee Transactions On Pattern Analysis And Machine Intelligence, 41, 4, 956-970
Year:
Apr 01 2019
Publisher:
Los Alamitos, IEEE COMPUTER SOC
ISSN:
0162-8828
1939-3539
Keywords:




 Record created 2019-06-18, last modified 2019-12-05


Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)