CamBall: Augmenting reality with a ball

April 29, 2011 Graphics

Using a camera as a direct human input device can be ideal for some types of applications, but it is relatively difficult and demanding to implement. The video feed, consisting of a huge amount of pixel colour values must somehow be filtered and interpreted robustly many times a second. CamBall is an experiment that does just that. It allows the user to interact freely with a virtual ball using only moderate hardware and a cheap webcam. It uses several known computer vision techniques to separate the captured user from the background, clean this up and feed it into a ball physics simulation. The simulated ball is then projected on top the video feed and presented to the user in real-time.

To separate the foreground from the background, a reference user-free image is first captured in CamBall's calibration phase. Once calibrated, the reference background image is continuously compared to the live images from the webcam at a steady 30 fps. Basically, only pixels that are different from the reference image are marked as foreground pixels. Obviously, webcam noise and slight changes to the lighting conditions seriously degrade the quality of the thresholded image. To counter this, a number of morphological operations are applied to close holes and filter out smaller thresholded areas. This type of segmentation can be quite successful, assuming the overall lighting conditions, the background and the webcam itself are kept static. Otherwise, regular recalibrations will be required.

Impulse collection

Impulse collection

The binary segmented video image is used as input to a soft-body ball simulator. Each frame, the intersection between the 2D circle (i.e. the ball) from the previous frame and the latest segmented foreground/background input image is inspected. For each foreground pixel inside the circle, an impulse towards the center of the circle is generated with a strength inversely related to the distance between the pixel and the center. To keep the ball inside the screen at all times, the screen border pixels are simply treated as foreground pixels as well. When all impulses have been collected, the ball position is updated using simple forward integration. Lastly, the ball is rendered on top of the live feed using this new position.

CamBall is a Windows application written in C++ back in 2006, using the OpenCV library for the segmentation steps. To see CamBall in action, see the video below, or try it out yourself using almost any (Windows compatible) webcam. If you run CamBall yourself, be sure to disable any automatic light-adjustment capabilities your webcam might have.

A nightly test run...


No comments yet.

Leave a comment