Inference of Segmented Volumentric Shape from Three Intensity Images

Parag Havaldar

Generating shape descriptions of objects form one or a few intensity images is an important problem in computer vision. Researchers have attempted to describe objects in terms of points, lines, surfaces and volumes. Among them, volumetric descriptions are the most descriptive, but are difficult to obtain from the images. This is because the objects are not directly given in the images, part of the object is facing away from the cameras and hence not seen in the images, surfaces may be partially occluded, etc. Under such circumstances, strong inference procedures need to be developed to recover the desired volumetric descriptions. Such descriptions provide compact representations which can be used to recognized objects, manipulate them, navigate around them and learn about new objects.

We use as input three intensity images of object from closely spaced viewpoints and attempt to generate volumetric and segmented (or part based) descriptions. The shape representation scheme we use to describe volumes are Generalized Cylinders (GCs). Past work (refs) in this area has concentrated on two subclasses of GCs - straight homogeneous generalized cylinders (SHGCs) and planar right constant generalized cylinders (PRCGCs). Projective invariant properties of these specific subclasses have been used to detect them in images. We show how a broader class of objects can be recovered by using properties of curved and straight axis GCs locally.

We start by extracting a hierarchy of groups from contour images in the three views. Grouping is based on proximity, parallelism and symmetry. The groups in the three images are matched and their contours are labelled as "true" and "limb" edges. These labels help in the inference of surfaces in 3-D, for example, the surface around a limb edge is locally smooth, where as around a true edge it is locally flat. We use the information about groups, the label associated with its contours (which could be all "true", all "limb" or a combination of both) to recover visible surfaces. We then use local properties of straight and curved axis generalized cylinders to obtain the position of the GC axis and the cross sections to make a volumetric inference. The final descriptions are volumetric and in terms of parts.

These descriptions are globally correct at the volumetric level but may still remain coarse at the surface levels where local dents and bumps might be present. The coarse volumetric descriptions obtained are then refined to include surface details as seen in the intensity images. We demonstrate results on real images of moderately complex objects with texture and shadows.