Activity and Event Recognition

Event Recognition

Many methods assume a segmented video and recognize actions within that segment, but for this project we require continuous event recognition. This involves both recognizing normal events and abnormal events, where an abnormal event may result in an alarm.

A Bottom-UP approach first detects and tracks objects and may also track limbs if detail is needed. Then the system applies activity models. A Top-down approach directly looks for evidence of a given activity activity. In this case, some bottom-up analysis (detection) may still be applied. Bottom-up systems more general but error-prone and slow. Top-down systems are more efficient and accurate if test examples conform to the detailed event models.

Previous work on recognition of complex events, given reasonable tracking and ability to recognize primitive events. Use of Hidden Markov Models and variations including Hidden Semi-Markov Models Demonstrated on tasks such as "stealing" (taking away an object) To develop ability to recognize larger library of complex events requires ability to recognize a large set of primitive events Focus of current work in event recognition

Even with motion capture data, action recognition is difficult due to the high dimensionality of the observation space. There are 67 joint positions and key features may be hidden in the high-dimensional space. Also the same action performed by different or even the same subject results in variations in the data.

22 Actions in 3 Categories and 3 Types

Event Representaion Languages

Our earlier projects developed a Video Event Representation Language (VERL) to represent events and a Video Event Markup Language (VEML) to annotate video.