High-Definition Web Demos with Macromedia Captivate, Flash, and Flash Media Server

Loren Leed

CollegeNet

Kenneth Toley

Adobe

In this tutorial, you will build a basic high-definition demo that you can deploy through the web using existing broadband technologies. A full in-house solution would require a team of developers with video production, ActionScript programming, and IT experience to stream content over your own Flash Media Server installation. Streaming services are also readily available, which allow longer program lengths without the need of an in-house server. Shorter program length can still function through progressive download from a standard HTTP protocol, although preload time can be considerably longer.

Flash developers with advanced ActionScript programming skills can follow the sidebar written by Adobe support engineer Ken Toley, and use it as a springboard for building full, rich media, high-definition demos.

Using this article, right-brained multimedia developers with at least intermediate video and Macromedia Captivate/Flash experience can use Ken's code like a black box and put together simpler demos without the aid of IT or software engineers. Be aware, however, that you will be working with some code and need to know enough about ActionScript to make the modifications required to get this project functioning with your own resources.

Requirements

To complete this tutorial you will need to install the following software and files:

Adobe Captivate

• Try
• Buy

Flash Professional 8 or Flash MX Professional 2004

• Try
• Buy

Flash Media Server (required for streaming files)

• Try
• Buy

Tutorials and sample files:

• cp_video_resources.zip (ZIP, 2.3 MB)
• cp_video_solution.zip (ZIP, 3.2 MB)

Background for This Tutorial

When I first saw Macromedia Captivate and Flash MX 2004, I wanted to be able to create sales demos for our company's software. I quickly began using Macromedia Captivate to record screens in our software as a way to automate the process of capturing and editing, but there was no easy way to get the content synced to the accompanying Flash video of our spokesperson, which was an important part of our project requirements.

Enter Adobe Flash Player quality assurance engineer Kenneth Toley, who took the potential of the two products and linked them with a few elegant lines of code. CollegeNET has successfully implemented this basic setup to create a series of demos that cover all the basic concepts of our scheduling software. Since launching the content in spring 2005, the demos have been viewed over 5000 times, freeing our sales staff to answer higher level questions and give more personal attention to each client.

Why Wait for Blue Ray DVDs When HD Delivery on the Web Is Here

Soon it will be possible to author high-definition (HD) content on the new DVD formats, but there will be few authoring tools and consumer players in this field. For now, consider the following web delivery option (see Figure 1):

1. Macromedia Captivate records simple Flash objects instead of endless bitmap frames to obtain pristine 800 x 600 screen captures in a remarkably small file footprint.
2. Flash video allows a real-time stream of video of a talking head at least 200 x 150 with very good quality picture and sound.
3. Place both in a Flash project to create the skin and you have a 1024 x 768 HD demo environment right in the browser.

Figure 1. The Macromedia Captivate.swf screen preloads and waits for resync script (1). The Flash video starts, utilizing the full broadband bandwidth (2). The interface skin can be over 1024 x 768 but generally should be slightly smaller to allow for the browser chrome on smaller laptop displays (3).

View Macromedia Captivate demonstration: High-Definition Web Demonstration

How the Solution Works, by Kenneth Toley

In the following section, Kenneth Toley, quality assurance engineer for Flash Player, explains how he created this solution.

When Loren contacted me for help, he had already recorded the instructional video (FLV) alongside the Macromedia Captivate recording. Being more of a programmer than a video producer, I think that Loren already completed the hardest part of the project. If you play back the FLV and the Macromedia Captivate SWF in ideal conditions, you can see that they were timed to function together. Five minutes of video playback corresponded to about five minutes of the SWF playback. The problem is that few users will be in ideal conditions, and to make the application interactive would require ActionScript that can adapt to the user and keep things synchronized.

Why All Is Not Synchronized by Default

Differences in bandwidth of the client machine, processor, and hard drive speed can all lead to differences in the start time of the Flash video file and the animation of the Macromedia Captivate SWF file. While you might always want the video and the animation to play at the same time, adding something as small as a false start of even half a second can really add up to a poor experience in a short time.

Subtle differences in performance across systems during playback can cause "hiccups" or short pauses in the animation or video as the client machine goes through its normal process of using RAM and the swap drive. Normally these hiccups occur at seemingly random times; on most systems they are rarely noticed, except when a few too many applications are open. However, when you are tying to get animation to synchronize with video, it is inevitable that the two will get out of sync at some point (on any system but the original system it is built on) no matter how good a client system you target. This is why some Flash producers like to use embedded video in their SWF files when synchronizing with animation.

Why Embedded Video Will Not Work

With embedded video, the video frames are basically converted to frame-by-frame Flash animations, ensuring that the animation and video are in sync with the playback time and performance of the Flash player on the client system. That works great with short videos or—even better—videos with no audio tracks, but with large videos this can quickly lead other performance limitations.

Separating the video from its audio track, which is what embedding does, means the video no longer keeps in sync with the original audio and you are back to square one with the synchronization issues. So what is the solution? In principle, you can use the same technique that the original video uses to stay in sync with its audio to keep the animation in sync with the video.

The Solution: Learned from Video

More or less, recorded video stays in sync with the audio because the video player knows which key frame in the video to be on at a which time during audio playback. At a regular interval, the video player compares the key frame it's showing to the frame it should be showing, and jumps ahead accordingly. When you do this enough times during playback, the result appears to be perfectly synchronized video and audio. Neat trick, huh? Well, for Loren's application, we needed to use a similar process with the Flash video and the Macromedia Captivate animation.

Making It Work

There are several things to work with. If you use the mediaPlayback component in Flash to play the FLV file, you can use the media API to indicate the state of the video while it is playing. You can listen to the component events to see if the user has paused, rewound, or fast-forwarded the content. You can also listen to ActionScript cue point events, which indicate the video's point in time during playback. If you know the frame that an animation should be on at that given time, you can move forward or backward in the animation to keep it synchronized with the video.

You can also stop and start playback of the animation in the SWF file to match the user controls of the component. All the properties, events, methods, and so forth that you need from the mediaPlayback component are in the Flash LiveDocs. The only thing you don't have is a list of frames that correspond to the points in time of the video playback mediaPlayback component. If you can put such a list into an XML file, you can load the XML file during playback and create the cue points that you must to listen to and synchronize with. Getting a good list of cue points for the XML file requires just a little bit of ingenuity.

Getting the Frames and Times for the XML File

A simple way to figure out the frame that the animation is on, given a point during the video playback for your XML, is to assemble the application and use an interval object to trace the current time of the mediaPlayback component and the current frame of the Macromedia Captivate SWF at regular intervals. About 8000 milliseconds or so, depending on your SWF frame rate, should be more enough to keep everything in sync. The lower the interval time, the more cue points you will have in your XML file. The more motion in your animation, the more cue points you may need.

Because in most cases it would be difficult to figure out how many cue points you will need and how close together they should be, this version of the application can make it easy for you to try different interval times to get it right, without having repeatedly to build your XML file by hand. Look at file examples in the BuildXML captivate_resources folder. The sample download files are structured in more or less the way you would expect the final content to be structured. The generated Macromedia Captivate FLA file and SWF file are named mastercal. The parent FLA and SWF file are named demo_skin.

The first thing to do is add a callback function that reads the playback time property from the mediaPlayback component and the current frame property from the loaded Macromedia Captivate SWF, and traces it out to the output window in a single line of XML. In the example, this code is on the CreateXML layer in the demo_skin example. Next, you add a small function to set the interval.

Note: The following code is temporary. After you get your XML, you can discard the following code in the final application.

/////
var intervalId:Number;  // interval 
var maxCount:Number = 100;
var duration:Number = 8.3; // milliseconds
var count2: Number = 0
//function to create a line of XML at each interval
function executeCallback():Void {

 if(myContent.captivate_mc._currentframe/100 >=1) {  // assume every 100 frames or so is a slide (optional)

count2 = Math.floor((myContent.captivate_mc._currentframe/100));
 } 
 
 //stop the interval when there is no animation left
 if (myContent.captivate_mc._currentframe == myContent.captivate_mc._totalframes){
      clearInterval(intervalId);
 }
 
// trace out the XML to the output window so we can save it to a file
 trace('<timeCode label="Slide'+ count2 +'" time ="' + kidvideo.playheadTime + '" frame = "' +  myContent.captivate_mc._currentframe + '"/> ');
}

function startIntervall(){

 intervalId = setInterval(this, "executeCallback", duration);
}
////

Next you must add some listeners to the mediaPlayback component so that you can start the video and the animation at the same time and call the startInterval function that will trace out your lines of XML. This is on the ActionScript layer of the demo_skin example:

////
var listenerObject:Object = new Object(); // object to attach our event methods to

listenerObject.load = function (eventObject){
        //once the video is loaded we can load the Captivate swf
    eventObject.target._parent.myContent.loadMovie('mastercal.swf');
}

listenerObject.click = function (eventObject){
    
    //when we click the play button on the video we can play the Macromedia Captivate SWF and start the interval function to generate the XML in the output window
         // start interval
    eventObject.target._parent.startIntervall();
    eventObject.target._parent.myContent.captivate_mc.play();
}

// Now we add the event listeners to the mediaPlayback Component
kidvideo.addEventListener("load", listenerObject);
kidvideo.addEventListener("click", listenerObject);
////

Now you need only to test the movie with the demoskin.fla file, select the play button on the mediaPlayback component, and wait for the video to play to the end. When it finishes, you can close the player window. You will see lots of XML in the output window. Select the menu on the output window and select Save to File. Save your file as timecode.xml. Now you have an XML file with the perfect data for generating the cue points.

At this point you can copy these files to a new folder called final and delete the CreateXML layer and the code you used. Now you can modify the demoskin.fla file so that it can use the XML file for the final application.

Bringing It All Together

You will need to modify the existing listeners in the demoskin.fla file and add several more to account for all of the states the video that could exist during playback while the user interacts with the application. You will also need to listen to the cue points during playback to keep the animation in sync.

Let's start by removing the call to the startIntervall function from the click listener event. Delete the following:

///
eventObject.target._parent.startIntervall();
///

Next, build the XML parser that loads and parses the XML as soon as the video loads:

///
//load cue points from an XML file.  

var myframe:Array = new Array();  // array of frames indexed by cue point name  we will need this to know what frame to jump the content swf to
    var myvid:XML = new XML();
myvid.ignoreWhite = true;
// here we load the cuepoint xml file and dynamically create our cue points and populate the array of frames
myvid.onLoad = function(sucess) {
    if(sucess){
    var boss = this.firstChild;
    while (boss != null) {
        kidvideo.addCuePoint(boss.attributes.label, boss.attributes.time);
        myframe[String(boss.attributes.label)] = String(boss.attributes.frame);
        boss = boss.nextSibling;
    }

    delete myvid;  // this saves runtime memory.. we don't need the xml object any more
    }else {
        kidvideo.removeAllCuepoints (); // if this fails for any reason for get it
        trace('no cues');
    }
trace('cue point xml loaded ' + sucess);  // we have loaded the XML file

};
///

To trigger this code when the video loads, change your load event listener to trigger the XML loading. Add the following code to the load listener above the action eventObject.target._parent.myContent.loadMovie('mastercal.swf'); as follows:

//
eventObject.target._parent.myvid.load('timecode.xml');
//

You can also set up some default variables to help keep track of the state. Add the following properties to your listener object right after instantiating it:

//
// the states we may want to check  and the default settings.
listenerObject.paused = false;  // is the flv paused
listenerObject.started = false;  //  has the flv started to play
listenerObject.scrubbing = false; // is the user moving the play head
listenerObject.lastState = null; // did the user scrub to a new time index
// all of the events
//

Next, add your cue point listener with several conditionals that will help you respond to the different states of the mediaPlayback component. This code controls the frame of the Macromedia Captivate SWF file when the user is scrubbing through the video or when it is just playing back.

Note: Scrubbing is the process of dragging the Player playhead to specific point to access to other parts of the Flash content. Since video went digital, users can use the slider or "slidebar" controls to navigate forward and backward through the video so they can play back the video stream or file from any specific point. Sometimes the slide control is called a scrub bar.

Add the following method to the listener object:

//
listenerObject.cuePoint = function(eventObject){
    //when we cross a cue point we need to update the content area
    trace ('cuePoint ' + eventObject.cuePointName +'called'); // we have crossed a cue point 
    var frame = eventObject.target._parent.myframe[String(eventObject.cuePointName)]
    
 // are we scrubbing or playing
    if(this.scrubbing){
        trace('scrubbing across');
            // if we are in a scrubbing state then lets just stop on the right content frame
        eventObject.target._parent.myContent.captivate_mc.gotoAndStop(frame);
        
    } else {
            // if we are not a scrubbing state then lets just play on the right content frame (we can't be out of sync)
        eventObject.target._parent.myContent.captivate_mc.gotoAndPlay(frame);    
    }
}
//

Because you don't want the animation running without the video, you must check for when the user pauses or stops the playback. Do so by adding a click method to your listener object. Modify the click method of the listener object accordingly:

//
listenerObject.click = function (eventObject){
    
    //whenever the user clicks the play button or slider the video pauses
    //and plays on release.  Check to see if it is playing and set the pause state
    // of the content SWF so we don't miss a beat.
    
    if (!eventObject.target.playing){
        
        this.paused = false;
    eventObject.target._parent.myContent.captivate_mc.play();
}else{
    this.paused = true;
    eventObject.target._parent.myContent.captivate_mc.stop();
}
trace('paused?  ' + this.paused );    
}
//

You also don't want the animation running while the user is scrubbing through the video, so you must pause the animation while the user is scrubbing. Add the following method to the listener object:

//
listenerObject.playheadChange = function(eventObject){
 trace('scrubing');  // the user is moving the play head
 this.scrubbing = true; // set state
 //stop content swf
 eventObject.target._parent.myContent.captivate_mc.stop();
 
 // here you can want make the content not visible but if you don't the user can
 // use the content as a point of reference
 
}
//

Next, you must be able to detect when the video starts and play back the Macromedia Captivate SWF file accordingly. Add the method to the listenerObject:

//
listenerObject.start = function (eventObject){
    this.paused = false;
    trace('start  ' + eventObject.target.cue points [0].frame);
    this.started = true;  // the presentation has started
    //Everything is ready when the video plays so should the swf
    eventObject.target._parent.myContent.captivate_mc.play();
}
//

If the user happens to scrub the video to a point between cue points, there is still the possibility to get out of sync before the next cue point. This is probably not a huge issue with this example because your cue points are close together and the Macromedia Captivate animation is fairly simple and short. However, if you have fewer cue points or a more complex animation, this will keep everything running smoothly, even if you have a user doing a lot of scrubbing throughout the video.

The resync function basically jumps the video to the nearest corresponding cue point before the point that the user stopped scrubbing. Everything is in sync by the time the most current cue point is triggered. Add the resync function:

//
// resync the presentation if the user scrubbed and played will not resync until we are playing
function resync(){
    // get the current time on the flv and find out what cue point is before it... and go there
    for (var x in kidvideo.cue points ){
        if(kidvideo.cue points [x].time < kidvideo.playheadTime){
            kidvideo.playheadTime = kidvideo.cue points [x].time;
            trace( ' goto time ' + Number(kidvideo.cue points [x].time) );
            break;
        };
    }
}
//

You must also add a start event to make sure you are in sync when the video starts to play.

Finally, the mediaPlayback component has a general change event you can exploit. Whenever the user interacts with the component, it triggers a change event. That means there is a good chance that several states of the component have changed as a result of the user interaction. This gives you an opportunity to make sure the state properties for listenerObject always reflect the correct state. Add the following methods to listenerObject:

// 
listenerObject.start = function (eventObject){
    this.paused = false;
    trace('start  ' + eventObject.target.cue points [0].frame);
    this.started = true;  // the presentation has started
    //Everything is ready when the video plays so should the swf
    eventObject.target._parent.myContent.captivate_mc.play();
}

listenerObject.change = function (eventObject){
    
    
        this.lastState = this.scrubbing;  // the state we are in before any changes happen will be our last state
        
//if we are playing in our current state then we are not scrubbing reset the state

    if (eventObject.target.playing){
 this.scrubbing = false;
 this.paused = false;

}
// If I was scrubbing in my last state but I am not in my current state and I am playing
// we need to call a function to move the play head to a time before the current most cue point
// that is the only way to stay in sync

if (this.lastState && !this.scrubbing && eventObject.target.playing){
 trace('call for resync');
 eventObject.target._parent.resync();  // call the function that resets our state from this point forward
}
}//

Now all you need to do is add the new listeners to your mediaPlayback component:

//
// Now we add all the event listeners
kidvideo.addEventListener("load", listenerObject); // this should already be in your code
kidvideo.addEventListener("start", listenerObject);
kidvideo.addEventListener("click", listenerObject);  // this should already be in your code
kidvideo.addEventListener("playheadChange", listenerObject);
kidvideo.addEventListener("cuePoint", listenerObject);
kidvideo.addEventListener("change", listenerObject);
//that's all!
//

Cross your fingers and test the movie. If all goes well, you will have synchronized your video with the Macromedia Captivate animation. If it does not work, check the version in the solution folder to help you track down the error.

Try Out the Solution

Follow these steps to apply Kenneth's solution to your project:

1. Within Flash, open demo_skin.FLA from the solution folder and select Control > Test Movie.

   Note: You can pause and scrub through the movie by grabbing the media component's playhead. The supplied sample files are set up to run locally. If you have access to a Flash Media Server streaming server, you can scrub through the entire movie; but with an HTTP web server, you can scrub only up to the point that's been downloaded.

2. Create or add your own resources:

   1. To create your own resources, export your screen capture from Macromedia Captivate as a Flash FLA file and export the video of your spokesperson from a video editing program as a Flash FLV file. Many options exist for creating these resources but they are beyond the scope of this article.

   2. To add your resources, open the demo_skin.fla file. Look at the ActionScript in frame 1. Edit this script by going to lines 69 and 70, where you will find commented instructions on where to specify the path names to add your own Macromedia Captivate content and timecode files.
3. Add the information for your FLV video directly into the Component inspector of the media component in Flash. Add the path/filename, length of time, and frames per second for the movie into the appropriate fields.

You can set up your Macromedia Captivate content in many ways and sizes. Just keep in mind that some things will start to consume bandwidth more than others. For instance, try to avoid full-motion capture because it forces Macromedia Captivate to abandon the "object-oriented" Flash style recording process, turning it into a less-efficient bitmap frame-field capturing machine like all the other screen recording products on the market. Do you really need to have full-motion scrolling during your demo at the cost of longer preload times? In most cases the answer is no.

We recorded the live video and the screen recording during the same session, but this requires a very fast computer with 2 GB of RAM and a large, unfragmented hard drive. The longest live capture we needed was 25 minutes on an IBM Pentium 4.

We also imported audio for the screen recording to make it easier to clean up editing in Macromedia Captivate, but we dumped it prior to the export to Flash. Be sure to make your video clips and Macromedia Captivate recordings the same total time to keep everything in sync. In Ken's resync code, the video FLV and the Macromedia Captivate SWF files must be the same time; otherwise drift can occur.

You can use the Macromedia Captivate Export Movie to Flash FLA command, which hides Macromedia Captivate and opens up Flash (which is required for the export process). Use the following steps to use the Macromedia Captivate Export Movie to Flash FLA command:

1. Import the Macromedia Captivate project file using the Import to Library option. Be sure to deselect the audio import option (remember, the audio portion is in the FLV file) and any other Macromedia Captivate feature that you did not use in your demo.

   Note: A large, high-resolution recording can take several minutes to import, and it's not always apparent that the import is occurring.

2. Upon import, open up the Library panel and locate the Macromedia Captivate content. It will have the same name as the original Macromedia Captivate project file but with a CPT extension.
3. Drag it to the Stage and name the instance captivate_mc. If you omit this step, the resync code will not work.

If you follow these steps and carefully examine the structure of the solutions files, your files should work properly with Ken's script. Even if you don't plan on creating your own code, read the instructions in the previous section, "Getting the Frames and Times for the XML File." Build the timecode generator version of your skin player, which is modeled after the sample files in the BuildXML Captivate_resources folder. This eliminates the lengthy process of entering all of the cue points into the XML file by hand, which is neither fun nor productive.

Lastly, the most common mistake is path errors, so make sure the project works locally with a simple directory structure before attempting to run it on a server. Once this basic setup works for you, you can take your Macromedia Captivate demos to a whole new level. Good luck!

Where to Go from Here

If you would like to have the option of streaming your video content in the future, test your FLV talking-head files on a Flash Media Server setup to make sure it streams. Some third-party streaming services have a free trial period where you can establish your compression settings from the beginning so that you do not have to reprocess your files later when you're ready to stream them. We settled for a conservatively sized 200 x 150 format for our spokesperson because even though the educational institutions have T1 connections, they are heavily used. Test your target audience and choose a compression scheme that makes sense for your application.

There is a practical limit for a movie length, which hovers around 15 minutes per section. A fast-moving Macromedia Captivate demo with lots of screen changes carries a heavier weight of bitmapped graphics for each changing background, and may not compress down far enough to allow for more than 7–8 minutes per section. The other important factor is the built-in 16,000 frame limit in Flash for SWF files. This imposes a ceiling cap on your timing.

A discussion of video production and establishing a Flash Media Server setup is beyond the scope of this article. For more information, visit the Flash Media Server Developer Center. I must say that once our project was up and functioning, it has been almost self-sufficient.

Developers who want to proceed without access to an advanced ActionScript programmer should not try to dream up a complicated interface for the Flash skin. Get the demo structure in this tutorial up and going with a bit of HTML, and then grow features over time. Test your proof of concept often, and at every level.

For more information about Flash video, visit the Flash Video Developer Center.

About the authors