A Computer Vision System for On-Screen Item Selection by Finger Pointing

Mi-Suen Lee, Daphna Weinshall*, Eric Cohen-Solal, Antonio Colmenarez and Damian Lyons

Philips Research USA, Briarcliff Manor, NY, USA
*Hebrew University of Jerusalem, Isreal


Pointing at planar surfaces such as computer and TV monitors or projection screens can be a useful mode of interaction between humans and machines. We describe below two designs where, using one or two cameras, the target of pointing on the flat monitor or screen is identified without 3D inference, using only image morphing and line intersection. When communicating with a computer, our users wear a wearable mouse (a thimble) whose color determines user ID as well as which user has ``control'' of the system at a given time. This device also simplifies finger tip detection. We describe experimental results showing a fully automated procedure for pointing target detection with high accuracy. The simplicity of our method and its robustness, as well as the relative accuracy of our results, can make pointing a practical means of human-machine interaction.

Paper in PDF format

Demo Videos

Video 1

Video 2