I s a a c  C O H E N 

Fusion of EO and IR Video Streams

Back to home

Students

Publications

Miscellaneous

Biography

D e s c r i p t i o n 

The problem of multi-sensors image sequences registra-tion requires selecting features that are invariant across sensors for a simultaneous alignment in spatial and tempo-ral domain. We use gradient information for extracting consistent features across different sensors.
In most of the cases, the temporal constraint is assumed to be known, and the spatial constraint is assumed to be constant. However, in this study, we did not have access to the calibration information of the rig and we had to register the geometrically the sensors using image features. Al-though the sensors are tightly mounted on rigid structure (e.g. camera rig), spatial variation can be observed frame by frame due to small errors in of the estimated perspective projection registering the views.
The geometric registration of the sensors is performed by a combination of a perspective and affine transforma-tions using the set of extracted invariant features. The per-spective transformation is estimated from the first frame of each sensor, and is used as an initial registration. The affine transformation is estimated from pairs of frames across sensors to estimate spatial drift between sensors. The opti-mal transformation is selected by measuring registration errors from both transformations.
The purpose of the sensor registration is the detection and tracking of the moving objects in the scene.

S t u d e n t s 

• Qian Yu
• Jinman Kang, graduated in 2004.

R e s u l t s 

P u b l i c a t i o n s 

• J. Kang, K. Gajera, I. Cohen and G. Medioni. Detection and Tracking of Moving Objects from Overlapping EO and IR Sensors, Joint IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum (OTCBVS’04). Washington, D.C. June, 2004. In conjunction with CVPR'04.