| Abstract |
This work aims to the automatic recovery of the 3D structure and the motion of a human hand from image sequences acquired by a single camera. A 3D geometric model of a hand is constructed using truncated cones, cylinders and ellipsoids. The 2D projection of the model is efficiently generated using tools from projective geometry. Hand tracking is then formulated as a state estimation problem. Hand model parameters define the internal state which is to be estimated from image observations consisting of edge maps and detected skin color.
Two approaches to the state estimation problem have been investigated. In the first approach, an unscented Kalman Filter (UKF) is used, following Stenger's approach [53], to update the model's pose based on local intensity edges and skin color. The resulting algorithm is capable of tracking smooth hand motion. However, manual initialization of the parameters of the hand model is required at the first frame, and no robust recovery strategy can be applied when track is lost. The second approach combines ideas from graphical models and particle filters to perform tracking based on nonparametric belief propagation (NBP) [59]. Using Monte Carlo methods, a general methodology is employed to recursively update particle-based approximations of continuous probability density functions. A graphical model describes the hand's 3D structure, kinematics and dynamics. The graph encodes hand pose via the 3D position and orientation of several rigid components, and thus provides local structure information in a global high-dimensional articulated model. Furthermore, NBP has been extended by embedding the mean shift mode detection algorithm [21] to accelerate the performance of the tracking process. The application of the NBP algorithm can recover the hand configuration robustly in the presence of outliers and local visual ambiguities.
Both developed tracking approaches are tested on several image sequences that depict rigid hand motion, pointing gestures and human hands engaged in object grasping activities.
|
Vision based tracking of a 3D model of a human hand
