Using 3D scene structure to improve tracking

Problem

• Aerial imagery collected over urban areas contains many occlusions for vehicles
• Current tracking algorithms cannot track vehicles through such occlusions due to weak object appearance and complexity of motion prediction over long period

Approach

• Use known 3D scene structure to estimate a dynamic occlusion map: binary map, which indicates what regions of the image are occluders of moving objects; dynamic due to camera motion
• Cluster track initiations and terminations near an occlusion map to find sources and sinks
• Find corresponding source-sink pairs using the Hungarian algorithm
• Match a sequence of tracks at the sink with a sequence of tracks at the source using dynamic programming
• Merge the matched tracks

Results

	Proposed Algorithm	Hungarian Algorithm
Correctly Linked	58%	42%
Incorrectly Linked	3.4%	5.8%

Data

• Database of 3D models (93 models, 40.5 MiB)
• Sequences used in experiments (83.9 MB)
• Updated occlusion maps (8.9 MB)

Code

Information coming soon!

Publications

• J. Prokaj and G. Medioni. Using 3D Scene Structure to Improve Tracking. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1337-1344, 2011.

  [PDF] [DOI] [Poster] [Video]

---