UAV Image Registration
Registering consecutive images from an airborne sensor into a mosaic is an essential tool for image analysts. Strictly local methods tend to accumulate errors, resulting in distortion. We propose here to use a reference image (such as a high resolution map image) to overcome this limitation. In our approach, we register a frame in an image sequence to the map using both frame-to-frame registration and frame-to-map registration iteratively. In frame-to-frame registration, a frame is registered to its previous frame. With its previous frame been registered to the map in the previous iteration, we can derive an estimated transformation from the frame to the map. In frame-to-map registration, we warp the frame to the map by this transformation to compensate for scale and rotation difference and then perform an area based matching using mutual information to find correspondences between this warped frame and the map. These correspondences together with the correspondences in previous frames could be regarded as correspondences between the partial local mosaic and the map. By registering the partial local mosaic to the map, we derive a transformation from the frame to the map. With this two-step registration, the errors between each consecutive frames are not accumulated. We then extend our approach to synchronize multiple image sequences by tracking moving objects in each image sequence, and aligning the frames based on the object's coordinates in the reference image.
Moving Object Detection on a Runway Sequence
To safely land any aircraft (whether manned or unmanned), it is essential to monitor the status of the runway prior to landing, regardless of the lighting conditions. We present here a method that aims at detecting moving objects on the runway from an onboard infrared camera during the approach. We make use of the fact that a runway is a planar surface. First, we locally stabilize the sequence with respect to automatically selected reference frames using feature points in the neighborhood of the runway. Next, we normalize the stabilized sequence to compensate for global intensity variation caused by the gain control of the infrared camera. We then create a background model to learn an appearance model of the runway. Finally, we identify moving objects by comparing each image in the sequence with the background model. We have tested our system on both synthetic and real world data and show that it can detect distant moving objects on the runway. We also provide a quantitative analysis of the performance with respect to variations in size, direction and speed of the obstacles.
Retinal Image Registration from 2D to 3D
We propose a 2D registration method for multi-modal image sequences of the retinal fundus, and a 3D metric reconstruction of near planar surface from multiple views. There are two major contributions in our paper. For 2D registration, our method produces high registration rates while accounting for large modality differences. Compared with the state of the art method , our approach has higher registration rate (97.2%vs. 82.31%) while the computation time is much less. This is achieved by extracting features from the edge maps of the contrast enhanced images, and performing pairwise registration by matching the features in an iterative manner, maximizing the number of matches and estimating homographies accurately. The pairwise registration result is further globally optimized by an indirect registration process.
For 3D registration part, images are registered to the reference frame by transforming points via a reconstructed 3D surface. The challenge is the reconstruction of a near planar surface, in which the shallow depth makes it a quasi-degenerate case for estimating the geometry from images. Our contribution is the proposed 4- pass bundle adjustment method that gives optimal estimation of all camera poses. With accurate camera poses, the 3D surface can be reconstructed using the images associated with the cameras with the largest baseline. Compared with state of the art 3D retinal image registration methods, our approach produces better results in all image sets.
Mutual Information Computation and Maximization Using GPU
We present a GPU implementation to compute both mutual information and its derivatives. Mutual information computation is a highly demanding process due to the enormous number of exponential computations. It is therefore the bottleneck in many image registration applications. However, we show that these computations are fully parallizable and can be efficiently ported onto the GPU architecture. Compared with the same CPU implementation running on a workstation level CPU, we reached a factor of 170 in computing mutual information, and a factor of 400 in computing its derivatives.
- Yuping Lin, Gérard Medioni, and Jongmoo Choi. Accurate 3D Face Reconstruction from Weakly Calibrated Wide Baseline Images with Profile Contours. Accepted in CVPR 2010.
- Yuping Lin, Qian Yu, and Gérard Medioni. Efficient Detection and Tracking of Moving Objects in Geo-coordinates. Accepted in Machine Vision and Applications.
- Yuping Lin and Gérard Medioni. Mutual Information Computation and Maximization Using GPU (pdf). CVGPU '08.
- Yuping Lin and Gérard Medioni. Retinal Image Registration from 2D to 3D (pdf). CVPR '08.
- Yuping Lin, Gérard Medioni. Map-Enhanced UAV image sequence Registration and Synchronization of Multiple Image Sequences (pdf) (ppt), Image Registration and Fusion Workshop '07.
- Cheng-Hua Pai, Yuping Lin, Gérard Medioni, Hamza, R.R. Moving Object Detection on a Runway Prior to Landing Using an Onboard Infrared Camera, OTCBVS Workshop '07.
- Yuping Lin, Qian Yu, Gérard Medioni. Map-Enhanced UAV Image Sequence Registration (pdf), WACV '07