In this paper, a 3D hand model fitting method is presented which can recover the accurate finger positions for a virtual keyboard system. The 3D hand model consists of a detailed polygonal skin driven by an underlying skeleton system. The system uses a structured light sensor to generate dense range measurements of user’s hand motion. We exploit depth information and match it against the model to estimate the pose of the hand. The parameters for model deformation are optimized with the guide of the applied forces between model points and range measurements. To speed up the optimization, we simplify the physical model and apply hash table- based fast point pair matching. The system can be used in any application requiring zero formfactor and requires no contact with a medium. Examples of applications include virtual reality, gaming, design, etc.