A Robust Processing Scheme for Layered Representation of Video Streams

Elaine Kang


Video contains more information than still images because it also captures the temporal evolution of scenes. Since the successive images in the video overlap by a large amount, it is desirable to have an efficient representation providing a significant reduction of size while including complete information. At the same time, it is desirable to extract elemental structural information that video contains, such as location of the moving objects, because many applications demand to manipulate video content. To address these goals, we adopt a multiple layer approach.

In our system, video sequences are represented by different layers: mosaic, background and independent motion layers. The mosaic layer provides the user a panoramic view and encodes camera motion. The background layer represents the static component of the video stream. The independent motion layers represent moving objects or shallow 3D surface structures at different depth. The background layer and independent motion layers are expressed in mosaic space. The layered representation can be directly applied for further processings, such as video surveillance and video compression. For compression, MPEG-4 also adopted a layered representation for video encoding, which shares the idea of our representation as a natural higher-level of the information encoding.

In this proposal, we present a robust processing scheme for realizing our layered representation of video sequences, from mosaic construction to independent motion layer extraction, We demonstrate our method on video sequences characterizing various applications.

Maintained by Philippos Mordohai