In our system, video sequences are represented by different layers: mosaic, background and independent motion layers. The mosaic layer provides the user a panoramic view and encodes camera motion. The background layer represents the static component of the video stream. The independent motion layers represent moving objects or shallow 3D surface structures at different depth. The background layer and independent motion layers are expressed in mosaic space. The layered representation can be directly applied for further processings, such as video surveillance and video compression. For compression, MPEG-4 also adopted a layered representation for video encoding, which shares the idea of our representation as a natural higher-level of the information encoding.
In this proposal, we present a robust processing scheme for realizing our layered representation of video sequences, from mosaic construction to independent motion layer extraction, We demonstrate our method on video sequences characterizing various applications.