Visual Hand Tracking and Gesture Recognition

Thomas Huang


We shall present some results of our research on hand tracking and gesture recognition in the last 5 years. This research is motivated by applications in human-computer interaction such as display control in virtual environments and the manipulation of virtual objects. Although we have studied both Appearance- Based and 3D Model-Based approaches, this talk will concentrate on the latter. At any given time instant, the hand configuration - 6 parameters for the global hand "pose" and 21 finger joint angles (the hand "posture") - is a point the 27-dimensional configuration space. We track the trajectory of this point over time using a 3D model-based and analysis-by-synthesis approach. The challenge is to represent the constraints on hand posture and finger movement in a compact way and use this representation to speed up the search in the 21-dimensional space. The tracking results can then be used to do gesture recognition. Some preliminary results will be shown on constraint representation and its use in tracking.

Maintained by Changki Min