An Investigation of Model Bias in 3D Face Tracking
Abstract
3D tracking of faces in video streams is a difficult problem
that can be assisted with the use of a priori knowledge of
the structure and appearance of the subject's face at predefined
poses (keyframes). This presentation provides an extensive
analysis of a state-of-the-art keyframe-based tracker: quantitatively
demonstrating the dependence of tracking performance
on underlying mesh accuracy, number and coverage
of reliably matched feature points, and initial keyframe
alignment.
Tracking with a generic face mesh can introduce an erroneous
bias that leads to degraded tracking performance
when the subject's out-of-plane motion is far from the set
of keyframes. To reduce this bias, we show how online refinement
of a rough estimate of face geometry may be used
to re-estimate the 3d keyframe features, thereby mitigating
sensitivities to initial keyframe inaccuracies in pose and geometry.
An in-depth analysis is performed on sequences of
faces with synthesized rigid head motion.
Subsequent trials on real video sequences demonstrate
that tracking performance is more sensitive to initial model
alignment and geometry errors when fewer feature points
are matched and/or do not adequately span the face. The
analysis suggests several indications for most effective 3D
tracking of faces in real environments.