Computer Vision for HCI and RTC Applications

Dr. Zhengyou Zhang


We strive to advance the state of the art of computer vision, and develop flexible and robust techniques for human-computer interaction and real-time communication and collaboration. In this talk, I will provide an overview of the research projects I have been working with my colleagues in these areas. I will cover the following topics:
* Face modeling with a webcam. We have developed a model-based face modeling system. A 3D face model is built in a few minutes, and the model can be animated immediately. We have successfully built 3D face models for Bill Gates, Steve Ballmer, and many others.
* Eye-gaze correction for video conferencing: The lack of eye contact in desktop video teleconferencing substantially reduces the effectiveness of video contents. We describe a novel approach: Based on stereo analysis combined with rich domain knowledge (a personalized face model), we synthesize, using graphics hardware, a virtual video that maintains eye contact.
* Whiteboard Technology: While physical whiteboards are frequently used by knowledge workers, they are not perfect. The content on the board is hard to archive or share with others who are not present in the session. We have developed a set of technologies which include automatic whiteboard note taking by scanning with a web cam and by enhancing the images, automatic audio and whiteboard meeting archiving and indexing, and live meetings with enhanced whiteboard streaming.
If time allows, I will also show two more prototype systems. The first converts an ordinary screen into a touch screen. The second converts a rectangular panel (e.g., an ordinary piece of paper) into a virtual mouse, keyboard and joystick.

Maintained by Adit Sahasrabudhe