Advanced Sensing and 3D Video Computing Techniques for Surveillance and Inspection

Zhigang Zhu


Recent advances in 3D computer vision and multimodal sensing technologies have made it possible to better protect an area or facilities through the integration of multiple sensors, from airborne imaging (for global 3D views), ground monitoring (for detailed 3D surfaces) to penetrating imaging (for 3D details inside) and remote listening (without microphones!). This talk will have two parts. In the first part, a new pushbroom stereo mosaic approach will be presented. The technique has been applied to four important surveillance and inspection applications: (1) three-dimensional (3D) reconstruction and moving target extraction via an airborne camera; (2) 3D scene presentation via a camera on a ground vehicle; (3) Under-vehicle inspection via a 1D camera array; and (4) 3D cargo inspection via gammar-ray vision. In the second part, I will present a multimodal integration approach for human detection and building inspection. The sensor modalities include a Laser Doppler vibrometer (for remote voice or vibration detection), a Thermal IR camera (with accurate temperature measurement) and a PTZ color camera (for human tracking or faade documentation). Video demos and experimental results will be shown. Joint work with George Wolberg, Robert Haralick, Weihong Li, Li Zhao and Hao Tang at the City University of New York; Jeff Layne at the Air Force Research Laboratory; Allen Hanson and Edward Riseman at the University of Massachusetts at Amherst. The work has been supported by AFRL, NSF, ARO, DARPA, NYSIA, CUNY and Atlantic Coast Technologies, Inc.

Maintained by Changki Min