IRIS-99-368

Tensor Voting for Salient Feature Inference in Computer Vision

Mi-Suen Lee

In computer vision, we often face the problem of identifying salient and structured information in a noisy data set. As computer vision systems move from controlled laboratory settings to real applications, the need for robust technique for inferring salient structures becomes more apparent. For a salient structure estimator to be useful in computer vision, it has to able to handle the presence of multiple structures, and the interaction between them, in any noisy, irregularly clustered data set. The derivation of such estimator relies on the proper implementation of constraints, particularly the continuity constraint.

We present a unified computational framework that make use of the continuity constraint to generate descriptions in terms of surface, regions, curves, and labelled junctions, from sparse, noisy, binary data in 2-D or 3-D. Each input site can be a point, a point with an associated tangent direction, a point with an associated tangent vector, or any combination of the above. The methodology is grounded on two elements: tensor calculus for representation, and non-linear voting for communication: each input site communicates its information (a tensor) to its neighborhood through a predefined (tensor) field, and therefore casts a (tensor) vote. Each site collects all the votes cast at its location and encodes them into a new tensor. A local, parallel routine such as a modified marching squares process then simultaneously detects junctions, curves and region boundaries. The proposed approach is very different from traditional variational approaches, as it is non-iterative. Furthermore, the only free parameter is the size of the neighborhood, related

We have develop several algorithms based on the proposed methodology to address a number of early vision problem, including perceptual grouping in 2-D and 3-D, shape from stereo, shape from shading, and motion grouping and segmentation, and the results are very encouraging.