Computational Study Of Nonverbal Social Communication

Dr. Louis-Philippe Morency, ICT, USC


The goal of this emerging research field is to recognize, model and predict human nonverbal behavior in the context of interaction with virtual humans, robots and other human participants. At the core of this research field is the need for new computational models of human interaction emphasizing the multi-modal, multi-participant and multi-behavior aspects of human behavior. This multi-disciplinary research topic overlaps the fields of multi-modal interaction, social psychology, computer vision, machine learning and artificial intelligence, and has many applications in areas as diverse as medicine, robotics and education.

During my talk, I will focus on three novel approaches to achieve efficient and robust nonverbal behavior modeling and recognition: (1) a new visual tracking framework (GAVAM) with automatic initialization and bounded drift which acquires online the view-based appearance of the object, (2) the use of latent-state models in discriminative sequence classification (Latent-Dynamic CRF) to capture the influence of unobservable factors on nonverbal behavior and (3) the integration of contextual information (specifically dialogue context) to improve nonverbal prediction and recognition.

Maintained by Dian Gong