Webcam Motion Detection: Using the BitmapData API in Flash 8

Some time ago I came across various processing experiments that monitored the active webcam for movement and allowed the user to interact with the program or art piece by moving around when in view. Until now Macromedia Flash Player has never been able to access the pixel data of a webcam feed; however, with the release of Macromedia Flash Player 8 comes an impressive and generous new feature—similar in many respects to Imaging Lingo in Director—that gives developers the power to create and modify bitmaps at runtime with ActionScript. This feature comes in the form of a new BitmapData ActionScript API.

One of my first experiments with Flash Player 8 was to try and re-create this webcam motion detection in an optimized fashion so that I could use this input as the basis for other webcam experiments, such as moving a ball around the screen by moving my hand around in view of the webcam.

Note: To view the example below, you need to have a webcam installed.

I set out on my journey into the realms of pixeldom and achieved the desired result. In this article, I discuss my journey and the outcome. By the end of this article, I will have taught you a little about how to use some of the new features available in the latest release of Flash Player and you should be able to use this knowledge to write some of your own experiments with webcams.

Since the release of Flash Player, you can capture the video stream from a webcam and display it inside a Flash Movie. All you need to do to accomplish this feat is place a Video symbol on the Stage and write a little ActionScript. It sounds complicated, but it is actually quite simple, follow these steps to try it for yourself:

1. Open the Library (press Control + L).
2. Create a new Video symbol from the Library menu in the top right-hand corner.
3. In the Video Properties dialog box, verify that the Video (ActionScript-Controlled) option is selected and click the OK button.
4. Go to the main Timeline and select the first frame.
5. Drag the Video symbol from the Library onto the Stage.
6. Select the Video symbol on the Stage then open the Property inspector (Control + F3).
7. Enter a unique name in the Instance Name field so you can talk to the video object with ActionScript. For now, enter output_vid for the instance name.
8. Open the Actions panel on the first frame of the main Timeline (F9) and add the following code:

9. That's it! Test your movie (press Control + Enter).
10. Flash Player asks the viewer for permission to access the webcam.
11. If permission is granted, the Video object you put on the Stage will display the video stream from the webcam.

In Flash Player 8, you can use the new BitmapData class to create bitmaps at runtime with ActionScript. This class holds individual pixel data and numerous methods are available to modify the pixel data inside of a BitmapData object.

One such method can be used to take a snapshot of a movie clip's current state as a bitmap object. You can also take a snapshot of the current state of a Video object, which is good news for me, because this experiment needs to access the individual pixels of the video captured from the webcam. To take a snapshot of the current state of a Video object or movie clip at runtime using ActionScript, use the draw() method of the BitmapData class.

You can then display a BitmapData object inside of a movie clip using the new attachBitmap() method:

You can then call your snapshot function, for example, when a button is pressed to display a freeze frame of the video object:

Note: Ensure that you do not have the video object at the upper left corner of the Stage, or the bitmap_mc instance will overlap your video.

To monitor changes in the video stream captured from the webcam, you need to take regular snapshots of the video object. I use setInterval to call a snapshot function every x milliseconds. Then, to actually determine whether there has been movement, at any moment in time two bitmaps have to be in memory—the bitmap created last time a snapshot was taken ("before") and a bitmap containing the current state of the video ("now"). Using the BitmapData class you can then compare the two bitmaps:

In my first attempt at this experiment I actually wrote some code that looped through every pixel in the current snapshot ("now") and checked for changes in the brightness of the pixel since the last snapshot ("before"). If the change in brightness of that pixel was greater than a constant tolerance value, then you can assume that there was movement in that pixel. The tolerance value is determined by trial and error, a value around 10 worked best in my experiments:

The results were satisfactory and it did actually work to some extent (see the old-webcam SWF file below), although speed was a big problem—the larger the dimensions of the video stream the more pixels I had to check. Even a small Video object (100 pixels wide, 100 pixels high) resulted in a large ActionScript loop that looped 10,000 times to check every pixel at a regular interval. Script timeout errors were common and the program appeared to lag.

So, my algorithm added size restrictions to the video feed from the webcam because I had to reduce the size of the ActionScript loop; 10,000 iterations is slow and unacceptable, so back to the drawing board I go. I decided that maybe I can speed up the loop if I only check the brightness of every ten pixels in the bitmap, and it worked. However, the effect lost accuracy and didn't detect all movement in the webcam. It was time for another solution that didn't involve checking the brightness of pixels.

This solution came about when a friend suggested that I should take a look at the difference blend mode, one of 11 blend modes available in Flash Player 8. There are nine blend modes in Flash Player 8 and two special blend modes that require extra steps for them to function correctly. A blend mode can be used to create a composite bitmap formed by blending together the color data from two or more bitmaps with different calculations. Each different calculation is a different blend mode. I was interested in the difference blend mode, because it creates a negative effect based upon the difference in brightness of the pixels.

You can apply a blend mode to a BitmapData object with ActionScript by specifying the name of the blend mode as a string for the fourth parameter of the draw() method:

The resulting bitmap contained a negative effect; any pixels that were the same color in both the previous snapshot ("before") and the current snapshot ("now") were black. Finally, a good result; I was almost there, but I still had an ugly negative image. The next point of call was to change all the pixels that were not black to a different color. In this case, I chose green, which resulted in a bitmap that showed only the movement detected in the webcam as a green outline.

You can change the color of any specific pixel in a BitmapData object using the setPixel() method. I could have used this method to check every pixel to see whether it is black (0xFF000000), and if it isn't, then I can change the color of the pixel to green (0xFF00FF00), as follows:

However, the larger the image, the more times this code will have to run. This could potentially slow down the Flash Player due to excessive loop iterations. Luckily there is another method of the BitmapData class called threshold() that you can use to isolate and replace ranges of colors in a bitmap. Because it is a native method, it is written in C code and thus it is lightning fast compared to the above code:

The above snippet of code basically instructs Flash Player to copy the pixels from the bitmap called now into the bitmap called now (itself), changing any pixels that have a color value that is greater than almost black (0xFF111111) to green (0xFF00FF00).

Where to Go from Here

Download the full source code and look at how everything works together in context to achieve the desired effect. I've commented the code to help you understand it. Visit Set Pixel and Flight404 for some inspirational webcam experiments—they should inspire you to develop some ideas of your own. Try and take this experiment to the next level, by using the motion detection to actually move things around the screen or to create interactive special effects. If you create something cool, send me an e-mail—I'd love to see it!