the

HandVu

vision-based hand gesture interface
overview | screenshots | requirements | download | installation | documentation | people

HandVu: Vision-based Hand Gesture Recognition and User Interface

With almost any color camera and sufficient processing power, this software collection implements a hand gesture interface. HandVu detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The output is accessible through a client-server infrastructure in a custom format and as OSC packets. Read more about HandVu or even download the software below

News

Overview

This software collection implements a vision-based hand gesture interface. Called HandVu, it detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The output is accessible through library calls, a client-server infrastructure in a custom format and as OpenSound Control packets, allowing control of music applications that use this format.
The software package consists of the main HandVu library and several applications that demonstrate video capture with OpenCV's highgui, DirectShow, and soon the ARtoolkit. If your application includes video processing, you would just make a few library calls and hand gesture recognition results will be available to you. If your application does not use camera input already, you can run one of the sample programs on the side and interface to your application with a networking interface. Functionality to capture screenshots and snapshots of the tracked object is availabe, allowing data collection for research or other purposes.

Screenshots

Videos are all in WMV format for now: The first one shows mouse-based interaction with a head-worn camera. The second one focuses on the hand tracking aspect in both indoor and outdoor scenes. The third video demonstrates interaction with virtual objects and two-handed interaction with help of the ARToolKit.

still images
interacting with virtual objects and ARToolKit
mouse and button interface with hand gestures
rubber band with-two handed interaction
gesturing to our wearable computer

Requirements

Download

Here's the installer for Windows: HandVu_beta4.exe
And this is the installer (package) for MacOSX: HandVu_Beta4.pkg.zip. Download the ZIP archive. It usually extracts automatically. If not, double-click the ZIP archive and extract the package that it contains. Double-click the package (PKG, or installer; it might be in your Downloads folder, or wherever the ZIP archive ended up) and follow the instructions. Once installed, you should be able to just go to the Applications folder and double-click on HandVu. Feedback appreciated!

Installation

Windows installer

On Windows XP, if you don't have Microsoft Visual Studio installed, then you first need to run the MSVS re-distributable installer: vcredist_x86.exe. These files come with Windows Vista by default. Then, execute the HandVu installer that you downloaded from above and follow the instructions. To run HandVu, double-click on either the HandVu_OpenCV icon or the HandVu icon on the desktop. For troubleshooting, see the Windows Installation documentation.

Apple Mac OS X Installer

THE INSTALLER IS NOT READY YET! If you know how to turn a command-line tool into a bundle (specifics for Info.plist file?), let me know! Once it's ready.... Download the DMG file from above. Double-click it. Drag the HandVu icon to the application folder. Run HandVu.

Documentation

Quick Start Guide

  1. Connect a camera to your computer.
  2. Run the desktop shortcut to HandVu_OpenCV or to HandVu. The live video will appear in the application window.
  3. Perform the closed posture at about the location shown below. After recognition of your hand, tracking will start and a number of key postures will be recognized.


Tijmen Kuipers kindly wrote an Installation Guide in Dutch

For more details, please have a look at the Application Documentation, which is included in the download package as well. Also, please make use of the Yahoo HandVu Group discuss installation or development problems.

Publications

Please see here.