Accessibility

Dynamic stream switching with Flash Media Server 3

David Hassoun

RealEyes Media
david.realeyes.com

This article provides an overview of the concepts involved in dynamic stream switching of video using Adobe Flash Media Server 3 (both Flash Media Streaming Server 3 and Flash Media Interactive Server 3). For an enhanced solution with Adobe Flash Media Server 3.5 and Adobe Flash Player 10, refer to my article, Dynamic streaming in Flash Media Server 3.5 – Part 1: Overview of the new capabilities.

Note: This article does not pertain to progressively downloading video content without the use of Flash Media Server.

I've included techniques and recommendations for bandwidth detection and the process to follow when preparing and encoding your video files for multi-bit rate delivery. The provided code examples demonstrate how to implement an effective mechanism for dynamically managing stream switching based upon the end user's bandwidth over time.

Flash is very powerful when it comes to working with video and streaming in general. It offers richness and functionality not found anywhere else on a platform (Flash Player) that is available to almost everyone. Even though it does not provide an out-of-the-box solution for dynamic stream switching, it includes all of the basic building blocks to enable almost anything with the right amount of development efforts.

With the readily available FLVPlayback component available in Adobe Flash CS3 Professional, as well as the VideoDisplay component in Adobe Flex 3, you have everything you need to quickly and effectively deliver video to your end users. Using ActionScript 3.0 you can extend and enhance the experiences using the provided components or by creating your own.

This article explores strategies to create a solution for dynamic bit rate switching that leverages the rapid development power of ActionScript 3.0 and addresses some of the unique challenges and benefits of using the Flash platform.

Requirements

To complete this article, you'll need to install the following software and files:

Flash Media Server 3 (or 3.5)

Flash CS3/CS4 Professional     or

Flex Builder 3 (with Flex 3.0 SDK)


Sample files:

Note: If you prefer, you can use your own video to follow along with this article, rather than using the provided video samples. You can use Flash Video Encoder (ships with Flash CS3 Professional), Adobe Media Encoder CS4 (ships with Flash CS4 Professional), or an alternate encoding option from Adobe Flash Media Solution Partners to encode your video media into three different bit rates.

Prerequisite knowledge

General experience working with video for Flash and Flash Media Server 3 is advised. Although some of the concepts are covered at an introductory level, much of what is presented is directed to intermediate or advanced developers.

Understanding the multi-bit rate solution

The goal of dynamically switching the stream to a different bandwidth during playback is to provide a better user experience for viewing large video files—including videos with extended length, large dimension, or very high quality—when the end user is in a network environment experiencing bandwidth fluctuations. If this scenario is not handled, the end user might experience a situation where video content can play for only a few seconds before it has to rebuffer. They may also end up watching a poorer quality video instead of the higher quality version available if more bandwidth had been freed.

Implementing a dynamic bandwidth switching process can enhance the end user's experience in two ways. First, it will initially calculate the bandwidth available between the server and the end user's machine and deliver the appropriate bandwidth initial stream. Additionally, it will continually monitor the stream and bandwidth and make adjustments on the fly, as needed. This method can effectively improve the video viewing experience without interrupting playback to an end user at any point.

One of the most difficult aspects of such a solution is making the switch between streams as seamless as possible. This may sound complicated, and it can be in varying bandwidth scenarios, as I'll discuss later in this article. Another primary concern in dynamically switching mid-stream to a different bandwidth is that if it is done too often, it can hurt playback more than it helps. After following along with this article you'll learn how this can be addressed and understand the challenges involved in determining factors such as the end user's average bandwidth over time in a streaming environment.

The challenge

More video is being streamed on commercial websites these days, and consumers have grown accustomed to seeing video-based content on the web. They associate these rich experiences with website and product quality. Additionally, the functionality available to share video-based content on the web has empowered a wider audience to become video contributors to the Internet. As a result, there is a lot of video content, and much of it is being delivered via Flash.

With this ubiquity of video—especially high-quality video—on the Internet, bandwidth consumption can be very high, even within standard use cases. Coupled with creative executions that specify, and server hardware that allows, streaming high-definition and full-screen content via the web, the incidence of an end user consuming their network connection's maximum bandwidth allowance is more common than ever. This issue can be intensified with standard network fluctuations that, if not handled properly, cause the end user to experience repetitive buffering of video content.

Requiring an end user to select a target bandwidth and associated video quality as the decision point for choosing which video bit rate to stream is not preferable, especially when this detection can be performed by Flash Media Server 3. Forcing the end user to make a choice not only results in poor user experience, but it is also likely that the end user's assessment will not accurately reflect the actual bandwidth available. Another major benefit of using the information you can determine about the connection between the client and the server is that you can also determine the state of the server. For example, even if the end user has an abundance of bandwidth, the server may be in a peak tasked capacity state, making the matching bandwidth on the server unavailable to deliver to the end user.

Monitoring bandwidth capabilities in a two-way scenario (server and client) is an important concept in stream management. Key factors in bandwidth fluctuation that can cause continual interrupted playback can generally be divided into three categories: server-side issues, connection provider (ISP) issues, and client-side issues.

Server-side issues

These issues can involve your Flash Media Server configuration, maximum server bandwidth or performance load exceeded, or content delivery network load balance or distribution issues.

Flash Media Server configuration
If your installation of Flash Media Server is not configured appropriately for the maximum simultaneous load, or if it has any specific application limitations that restrict the number of connections and/or allocated bandwidth, these factors could adversely affect user experience and could lead to bottlenecking connections. In general, if the maximum capacity has been reached at the server to deliver the appropriate experience to each connected end user, the server should not allow any additional connections until a previous user has disconnected. This configuration ensures a quality experience for the already connected users.

Maximum server bandwidth or performance load exceeded
Similar to the issue described above, it is recommended that the number of connections be limited so that a high-quality experience is maintained for existing users. Setting a limit on the number of connections is optimal, compared to letting an unlimited number of users connect to the server. This setting will be based upon your specific application requirements. You can avoid playback issues by performing proper load planning calculations and monitoring actual server usage.

CDN load balance or distribution issues
If you are using a content delivery network (CDN) to deliver your streaming content, any balance or distribution errors on the CDN's network will result in issues for your users. Bandwidth allocations and connection limitations are generally part of your agreement with a CDN vendor. Be sure appropriate planning is applied to handle peak and burst scenarios in order to achieve the best end user results.

Connection provider (ISP) issues

These issues can involve dynamic bandwidth throttling or networks affected during peak user times.

Dynamic bandwidth throttling
Some Internet Service Providers (ISPs), as well as some more advanced network management scenarios such as educational, government facilities, or corporate network environments will integrate a dynamic bandwidth throttling network infrastructure. The purpose of the dynamic throttling is to attempt to provide all users with a high-quality experience by limiting those with high session consumption rates over time. The end result for these users is a fast initial connection with plenty of bandwidth available; however, the bandwidth allocation for users consuming great quantities of bandwidth for extended periods (such as streaming longer video content or downloading files from a peer-to-peer network) is continuously decreased to allow new connections to access more initial burst bandwidth and to generally discourage extended use. Dynamic throttling can be difficult to manage in normal scenarios, especially when streaming HD video.

Peak user time affecting network
Some types of network/Internet connections are greatly affected by the gross number of concurrent users on a large network at the same time. This issue occurs most frequently on cable Internet connections. If a significant number of end users are connected simultaneously, you may notice greater fluctuations and decrements to the available bandwidth during the course of a connection.

Client-side issues

These issues can involve shared connectivity, network hardware issues, or client hardware limitations.

Shared connectivity
Users in a shared connectivity environment—such as a household, Internet café, WiFi hot spot, airport, or small office—share the relatively limited bandwidth among the other simultaneously connected users. When many concurrent users connect to a small network, it is likely that the experience for any one end user will fluctuate a great deal. In addition, if there are any "power users" on the network who are consuming a relatively large amount of data for a single end user, (such as multitasking, watching videos, uploading content, using an online collaborative application, talking on VoIP, or using a web cam) the shared bandwidth across the network can become increasingly slow.

Network hardware issues
Although generally less common, client-side hardware issues with routers, hubs, or network interface devices, or other problems such as driver conflicts can also lead to bandwidth issues for end users.

Client hardware limitations
On the client side, it's important to consider the limitations caused by the user's hardware, such as their video card. If their video card can't process the incoming information fast enough or if the system's resources are otherwise occupied, the stream's buffer may empty. In addition to hardware limitations, standard fluctuations in connections should be expected. These fluctuations can have adverse affects on the playback of video content. This issue is especially encountered when streaming HD content, especially in the full-screen aspect. In such cases, users typically have the vast majority of their bandwidth committed to sustaining the stream, which leaves little margin for correcting any other factors such as normal bandwidth fluctuation. Videos with shorter duration, smaller file size, and smaller dimensions generally are not as susceptible to bandwidth issues and fluctuation. Therefore, determine if the video content you are streaming warrants the implementation of the tactics outlined below. If you are not streaming HD content, some of these strategies may not be necessary.

Later in this article I'll discuss bandwidth-limiting issues further and describe best practices for handling them directly. You will learn how to take the solutions further and customize them to suit your specific needs.

The recommended solution

Dynamic bandwidth stream switching can be implemented in many ways. Each approach has its own benefits and consequences. The single solution provided here is a direct and viable one that provides high accuracy and minimizes end user disruption. Remember that it is important to consider the requirements of your particular video distribution system before implementing any of these solutions. The following questions can help you determine the best approach to take:

After analyzing your system and identifying a need to implement a dynamic bandwidth switching solution for your application, you can research how the solution will work and how to implement it. The diagram below illustrates the flow of the recommended solution (see Figure 1).

Flow chart of the dynamic stream switching solution

Figure 1. Flow chart of the dynamic stream switching solution

The flow of the dynamic stream switching solution can be described as follows:

  1. The application initializes. It accesses a collection of video files and their matching bit rates.
  2. Once the connection to Flash Media Server is complete, perform a bandwidth detection test.
  3. Based on the results from the bandwidth test, start the appropriate stream from the collection.
  4. Start playing the video with a small, quick start buffer.
  5. Once the quick start buffer is full, set the buffer to a standard size,
  6. Start listening for bandwidth issues and the need to step-down the bit rate by monitoring the buffer empty events and their count versus the threshold.
  7. Once the standard buffer size is full and there have been no issues, start a timer to check that playback is uninterrupted and smooth for a reasonable duration (approximately 20–30 seconds) before starting the upscale test. If any buffer issues are triggered during the standard extension test, start over by returning to step five.
  8. If the standard play test was successful, track the current size of the filled buffer, and also track the current time in the application (not the video). Set the buffer time to 2–3 times its current filled length (be sure to use its current length which includes the buffer overrun value—rather than the original buffer length).

  9. Wait for the new extended buffer to be filled. Once it is full, calculate the available bit rate using these formulas:

    [buffer amount loaded in test] = [target extended buffer] – [tracked start buffer time]

    [time to complete test] = [current time of application] – [tracked start time of test] (convert to seconds)

    [total bytes loaded in test] = [current stream bit rate] × [buffer amount loaded in test]

    [available bit rate] = [total bytes loaded in test] ÷ [time to complete test in seconds]

  10. If the available bit rate is enough to upscale to the next stream, swap to that stream and restart the process from Step 4.

Step 1: Encoding for multi-bit rates

In the previous section I described how a multi-bit rate solution could be implemented to enhance the end user's video experience. In this part, we'll begin working with the sample video files. To follow along, you'll need multiple versions of the same video content to stream. Download the sample files provided on the first page of this article, or locate your own video files on your hard drive. You can use the Flash Video Encoder (which ships with Flash CS3 Professional) or Adobe Media Encoder CS4 (which ships with Flash CS4 Professional) to encode the video at different bit rates. Alternately, you can also use other video encoding options that are described at the end of this section.

One of the primary requirements to providing a clean solution for dynamic bit rate stream switching is having a set of videos of different bandwidths available from a single source. This will enable us to determine which file is best suited for the end user at the different bandwidths they may connect at or shift to over time. The key factors we need to manage between the videos are as follows:

Let's get started by preparing some video for streaming. We'll encode videos for several different bit rates:

Ensure that all of the encoded videos use the same frame dimensions. We'll be switching between the videos as they're playing, and the transitions will be smoother if all of the videos in a set have the same frame dimensions.

Encoding walkthrough

The sample video source is encoded as a High-Definition QuickTime MOV file using the H.264 MPEG compression at an approximately 5 Mb/second bit rate encoding and AAC audio. The source file has the frame dimensions of 1280 × 720 and a frame rate of 24 fps. It is most definitely not a web ready video, but it suits our purposes well as the master source file to use when encoding the web versions we'll deliver.

It is important to note that whenever you are encoding your own video you should always strive to use the highest quality, least compressed source to achieve the best results. In this walkthrough we will use the bit rates listed above to encode the web-quality video clips in preparation for setting up the dynamic bit rate swapping in the later exercises:

  1. Open Flash Video Encoder. Click the Add button to add a video file to the queue.
  2. Navigate to the AdobeMax2007_720p.mov file in the provided sample files folder (or choose your own high quality video source) and click Open.
  3. Select the video listed in the queue and click the Settings button. The Flash Video Encoding Settings dialog box appears.

  4. Set the Output filename to AdobeMax2007_720p_150. Switch to the Crop and Resize tab. Check the Resize video checkbox, and then set the height to 240. The width should automatically resize to 426 (see Figure 2).

    screenshot_FlashVideoEncoder1.jpg

    Figure 2. Select the option to resize the video and set the height to 240 pixels

  5. Switch to the Video tab and make sure that the frame rate is set to Same As Source, which is the default setting.
  6. Set the Max Data Rate field to 150 kilobits per second. Leave all other settings as the default settings, and then click OK.
  7. In the queue, select the video you just modified. Click the Duplicate button. This will add another video to the queue.
  8. Select the duplicated video and click on the Settings button. Change the Output filename to AdobeMax2007_720p_300 and then switch to the Video tab and change the Max Data Rate to 300 kilobits per second. Click OK.
  9. Repeat Steps 7 and 8, creating videos at 500, 700, and 1500 kilobits per second, changing the Output filename for each to reflect the corresponding Max Data Rate (see Figure 3).
  10. Once you've added all these videos to the queue, click the Start Queue button. Flash Video Encoder will process all the files and output them to the same directory as the source video file.

Duplicate and add settings to create three more versions to encode

Figure 3. Duplicate and add settings to create three more versions to encode

That's all you need to do to encode the video sample files for this article. Keep in mind that there are other settings you can tweak later to provide your users with more viewing options. You can encode videos at different levels of smoothing or at different frame sizes. Although switching between videos with different settings for smoothing and frame sizes is outside the scope of this article, you may wish to experiment later with switching between videos using these settings as well.

Other encoding options

Adobe works with partner companies to provide Flash Media Server assistance, including video encoding services. Encoding partners can set up solutions such as server-based transcoding or simpler needs, such as batch encoding of media files. Table 1 includes a list of Adobe partners and the type of Flash Media Server services they provide. To learn more, see the Flash Media Server: Ecosystem partners.

Table 1. Flash Media Server services of Adobe partners

Publishing Encoding Ad delivery Consulting
Brightcove Anystream PointRoll Digital Primates
DBee Digital Rapids EyeWonder Fig Leaf Software
ExtendMedia Envivio Inc. Unicast Intesolv
The FeedRoom Inlet Technologies VoloMedia New Toronto Group
Jalipo Kulabyte YuMe Networks RealEyes Media
Maven Networks Media Excel SAIC
Multicast Media On2 Technologies TCi
Onstream Media RipCode Universal Mind
stimTV Sorenson Media
Streamedia Telestream
TANDBERG Television ViewCast
thePlatform

Step 2: Bandwidth detection

The bandwidth detection phase is generally a single run operation that occurs at the initialization of the application. The detection data is used to select the initial bandwidth associated with a video file. If the measurement taken during this phase is accurate, then most end users will be initialized with the proper stream. Unless a bandwidth-limiting event occurs, it won't be necessary to switch streams.

A Flash application by itself is unable to perform bandwidth detection, because Flash Player doesn't include a built-in mechanism to handle bandwidth detection. There are a couple of reasons for this. In order to complete a bandwidth test operation, Flash Player would require a separate source on the server to test against. The standard procedure for testing bandwidth involves measuring the amount of time it takes to load a certain amount of data. Additionally, if the bandwidth detection test was performed between the web server hosting the SWF file and the user instead of testing between the user and Flash Media Server (which is often a different server) the resulting test could be less accurate.

Recommended approach

In general, testing the bandwidth between the streaming server and the specific client is the best strategy because it will identify the optimal current bandwidth either end can support. When using Flash Media Server 3, bandwidth detection has been simplified and is available by default for new applications created server-side. This testing functionality can be configured; refer to the Flash Media Server documentation for help with restricting it.

Bandwidth detection walkthrough

Determining bandwidth is the first step in setting up bandwidth switching. In this walk-through we'll set up a simple application that connects to an FMS3 server and then checks the client's bandwidth (see Figure 4).

Step 2 of the flow chart identifies the initialization phase, when available bandwidth is detected

Figure 4. Step 2 of the flow chart identifies the initialization phase, when available bandwidth is detected

Set up the files

Follow the steps below to prepare the files for the bandwidth test:

  1. Copy the BandwidthTest sample files to an appropriate location on your computer.
  2. To create the FMS application that you will connect to from the server, locate your Flash Media Server install directory and find a folder inside the applications directory called VOD. This is where the FMS application will live, and you'll use it to complete the bandwidth tests.
  3. Open the BandwidthTest.fla file.
  4. Open the BandwidthTest.as file in the com.bitratesample.document package. This is the Document class file for the BandwidthTest.fla.
  5. Replace the value set for rtmpPath on line 31 with a valid FMS server path. If you're using Flash Media Development Server 3 on your local computer, replace www.yourfmsserver.com with localhost: 
    public var rtmpPath:String = "rtmp://www.yourfmsserver.com/VOD/";

Now the application is set up and it's ready to connect to the server.

Create a client for the NetConnection

Now we'll need to create a custom client for our NetConnection:

  1. Open SimpleClient.as in the com.bitratesample.net package.

  2. The SimpleClient class should have two methods: onBWCheck() and onBWDone().

    The method onBWCheck() is required for the bandwidth check to work and is provided for you.

    The method onBWDone() allows us to handle the result of the bandwidth check. We will need to add code to retrieve the result of the bandwidth test to this method.

  3. Under the comment //Create a local bandwidth variable, create a variable named bitrate that is typed as a Number. Set the value of the bitrate variable equal to the first index in the rest array. Your code should look similar to the following: 

    // Create a local bandwidth variable
var bitrate:Number;
// Get the bandwidth value from the rest array
bitrate = rest[0];

  4. This next part alerts the application that we have a bandwidth value. Under the comment // Dispatch an event and pass the bandwidth value, dispatch a new BandwidthEvent. The first parameter (type) should be BandwidthEvent.BANDWIDTH_RECEIVED, and the second value should be the bitrate variable. The code should look like this:

    // Dispatch an event and pass the bandwidth value
dispatchEvent( new BandwidthEvent( 
BandwidthEvent.BANDWIDTH_RECEIVED, bitrate ) );

Set up the bandwidth test

  1. Return to the BandwidthTest.as file in the com.bitratesample.document package.
  2. In the constructor, under the comment // Create the client, set the _client property equal to new SimpleClient().

  3. We need to listen for the BandwidthEvent that will be broadcast from the onBWDone() method in the SimpleClient. Add an event listener on the _client for the BandwidthEvent.BANDWIDTH_RECEIVED. The event handler method should be _onBandwidthReceived. The code should look like the following: 

    // Create the client
_client = new SimpleClient();

// Add the listener for the bandwidth result
_client.addEventListener(
BandwidthEvent.BANDWIDTH_RECEIVED, _onBandwidthReceived );

  4. Now we need to create a NetConnection Object. Set the client property of the _nc NetConnection Object equal to _client, like this: 

    // Create our net connection
_nc = new NetConnection();


// set the client on the NetConnection
_nc.client = _client;

  5. Beneath the comment // Create our listeners for the NetConnection, create an event listener for the NetStatusEvent.NET_STATUS and assign the handler method of _onNetStatus:

    _nc.addEventListener( NetStatusEvent.NET_STATUS, _onNetStatus );

  6. Call the connect() method on the _nc NetConnection Object, being sure to pass rtmpPath as the method's first parameter and true as the second parameter. The VOD (Video On Demand) application is now included with the standard Flash Media Server installation. When you pass true as the second parameter, it specifies to the VOD application that the bandwidth check should start as soon as the application connects.

    Note: If you are not using the VOD application you do not need to pass true as the default second parameter to connect. Instead you'll need to invoke the bandwidth test manually by calling the checkBandwidth method on the NetConnection class after it successfully connects.

    // Connect
_nc.connect( rtmpPath, true );

Your constructor should look similar to the following:

public function BandwidthTest()
{ 
// Create the client
_client = new SimpleClient();

// Add the listener for the bandwidth result
_client.addEventListener( BandwidthEvent.BANDWIDTH_RECEIVED,
_onBandwidthReceived );

// Create our net connection
_nc = new NetConnection();

// set the client on the NetConnection
_nc.client = _client; 

// Create our listeners for the NetConnection
_nc.addEventListener( NetStatusEvent.NET_STATUS, _onNetStatus );

// Connect
_nc.connect( rtmpPath, true );
}

Handle the events

In this section we'll set up the handlers to perform bandwidth detection in the application:

  1. In the event handler method _onNetStatus() locate the case statement for "NetConnection.Connect.Success". This is the point at which we know that we are connected to the server and can start our bandwidth check.
  2. After the comment // Now we can check the bandwidth, add the following code to initiate the bandwidth sequence:
    // Now we can check the bandwidth
_nc.call( "checkBandwidth", null );
  3. The _onBandwidthReceived() method has been created for you and outputs information to the screen. Save all the files and test the application.

You should receive output similar to the following:

p_event.info.code: 
NetConnection.Connect.Success
The connection attempt succeeded.
Now we know our bandwidth!
Bandwidth: 9854

Figure 5 shows a screenshot displaying the above information.

Connection and bandwidth information of the application

Figure 5. Connection and bandwidth information of the application

Step 3: Bit rate selection

Bit rate selection is the process of determining the initial bit rate encoded file to play, based on the data obtained in the bandwidth detection phase detailed above. Once you have detected the initial bandwidth available between the client and Flash Media Server you can use that number to select the corresponding highest quality (bit rate) file in your encoded set to play back. In order for the bit rate selection process to work successfully, you must already have the bit rate data and the path to each file within the series that you are playing back.

Recommended approach

After performing a standard bandwidth detection operation, use the available bandwidth number and cross reference it against your video list. Select the video that has an encoded bit rate less than the result value from the bandwidth test. Generally speaking, you should be careful not to select a file that is too close to the total available bandwidth, because doing so may put you in a high-risk scenario for issues. Developers will often set a margin of error as a fluctuation bit rate value, and this number may subtracted from the bandwidth test result before performing the cross reference of video files, as an added safety precaution.

Note: A best practice for clean management and processing is to keep track of the video bit rate separately from the paths to the video files.

Bit rate selection walkthrough

In this walkthrough, we'll manage a set of videos encoded at their respective bit rates to cover the full spectrum of standard bandwidth usage. After the standard bandwidth detection is complete, we'll use the set of videos to select the best file to play in the video stream with the optimal bit rate selection (see Figure 6).

flow_diagram_BitrateSelectionWalkthrough.jpg

Figure 6. After performing bandwidth detection, the next step is to determine which stream to use

Set up the files

Follow the steps below to prepare the files for the bit rate selection operation:

  1. Copy the SelectStream sample files to an appropriate location on your computer.
  2. Open SelectStream.as in the com.bitratesample.document package.

  3. Update the rtmpPath property to point to a valid FMS server. If you're using Flash Media Development Server 3, just change www.yourfmsserver.com to localhost:

    public var rtmpPath:String = "rtmp://www.yourfmsserver.com/VOD/";

Create the list of streams to play

In this part we'll create an array of the video files in order to select the best one to play:

  1. Locate the _createStreamList() method. This method is called from the _init() method and we'll use it to create a list of objects containing stream information.
  2. Under the comment that begins // Create an array of objects, set the property _streamList equal to new Array().

  3. We need to add objects to the array that contain the path and bit rate of the streams we want to play. For each of the items in the following list, push a new object into the array with the following properties:

    1. Path: AdobeMax2007_720p_150 – Bit rate: 150
    2. Path: AdobeMax2007_720p_500 – Bit rate: 500
    3. Path: AdobeMax2007_720p_700 – Bit rate: 700
    4. Path: AdobeMax2007_720p_1100 – Bit rate: 1100
    5. Path: AdobeMax2007_720p_1500 – Bit rate: 1500

  4. Be sure your code looks like the following:

     // Create an array of objects that contain two properties bitrate and path
 _streamList = new Array();
 _streamList.push( { path:"AdobeMax2007_720p_150", bitrate:150 } );
 _streamList.push( { path:"AdobeMax2007_720p_500", bitrate:500 } );
 _streamList.push( { path:"AdobeMax2007_720p_700", bitrate:700 } );
 _streamList.push( { path:"AdobeMax2007_720p_1100", bitrate:1100 } );
 _streamList.push( { path:"AdobeMax2007_720p_1500", bitrate:1500 } );

  5. Under the comment // Store the length of the array, set the value of _streamListLength equal to _streamList.length:

     // Store the length of the array
 _streamListLength = _streamList.length;

  6. Under the comment that begins // Sort the streams by descending bit rate, we'll sort the Array.DESCENDING on the bitrate value. This will make it easier to determine the best quality to play later on:

     // Sort the streams by descending bit rate so we can grab the appropriate bit rate when needed
 _streamList.sortOn( "bitrate", Array.NUMERIC | Array.DESCENDING );

The _createStreamList() method should look like the following: 

 private function _createStreamList():void
 {
 // Create an array of objects that contains two properties bitrate and path
 _streamList = new Array();
 _streamList.push( { path:"AdobeMax2007_720p_150", bitrate:150 } );
 _streamList.push( { path:"AdobeMax2007_720p_500", bitrate:500 } );
 _streamList.push( { path:"AdobeMax2007_720p_700", bitrate:700 } );
 _streamList.push( { path:"AdobeMax2007_720p_1100", bitrate:1100 } );
 _streamList.push( { path:"AdobeMax2007_720p_1500", bitrate:1500 } );
 // Store the length of the array 
 _streamListLength = _streamList.length;
 // Sort the streams by descending bit rate so we can grab the appropriate bit rate when needed
 _streamList.sortOn( "bitrate", Array.NUMERIC | Array.DESCENDING );

Set up the NetStream

In this section of the walkthrough we'll create a new NetStream and then attach it to the video object:

  1. Locate the _onNetStatus() event handler method.
  2. Under the comment // Set up the NetStream in the NetConnection.Connect.Success case, set _ns equal to new NetStream(), making sure to pass in the NetStream constructor _nc.

  3. On the _ns NetStream object, add an event listener for AsyncErrorEvent.ASYNC_ERROR, and the event handler method _onAsyncError() like this: 

     // Set up the NetStream
 _ns = new NetStream( _nc );
 _ns.addEventListener( AsyncErrorEvent.ASYNC_ERROR, _onAsyncError );

  4. Attach the NetStream to the Video object under the comment // Attach it to the video object:

    // Attach it to the video object
video.attachNetStream( _ns );

Determine the highest bit rate file to play

This next part involves adding the selection operation, in order to choose the appropriate file to play using the bandwidth detection data:

  1. Locate the _onBandwidthReceived() method. We'll add an if statement inside this method that determines the best stream to play (based on the bandwidth data received) and then we'll set it to play that stream.

  2. Under the comment that begins // Now that we know our bandwidth, create an if() condition to evaluate if p_event.bandwidth is greater than 0 (zero):

     // Check to make sure we have bandwidth - if p_event.bandwidth > 0 then select the stream and play it
 if( p_event.bandwidth > 0 )
 {
  }

  3. If p_event.bandwidth is greater than 0 (zero) create a local variable named videoIndexToPlay typed as an int and set it equal to the result of a call to _getBandwidthIndex(). We will add logic to _getBandwidthIndex() in a moment. Make sure you pass p_event.bandwidth as the only parameter:

     var videoIndexToPlay:int = _getBandwidthIndex( p_event.bandwidth );
  4. Locate the _getBandwidthIndex() method. Find the line that creates the idx variable (this will be the variable that we return). Add the logic to loop though the _streamList array and set idx equal to the current index in the loop if the bitrate value of the current item in the _streamList is less than or equal to the p_targetKbps value that was passed into the _getBandwidthIndex() method. We'll also multiply our target bit rate by our safety margin. Here's the code: 
     var idx:int;
 for( var i:int = 0; i < _streamListLength; i++ )
 {
 if( _streamList[i].bitrate <= p_targetKbps * _safetyMargin )
 {
 idx = i;
 break;
 }
 }

Note: Make sure you add a break; once idx is set so we get the highest possible stream based on the bandwidth available.

Play the stream

After setting up the play command, we can test the application to see how it works:

  1. Return to the _onBandwidthReceived() method. In the if() condition, under the declaration for videoIndexToPlay, call the _startStream() method while passing it the correct Video object from the _streamList array using the videoIndexToPlay variable:

     _startStream( _streamList[videoIndexToPlay] );

  2. Locate the _startStream() method. Under the comment // Play the video, call the play() method on the _ns NetStream object and pass it the path property of the p_newVideoObject parameter, like this: 

     // Play the video
 _ns.play( p_newVideo.path );
  3. Save all of the files and open UpScaleDownScale.fla. Test the movie to see the video stream play and the status appear.

At this point in our development, you should see output in the text field on the right side of the screen and the initial video should begin to play on the left. We're not quite finished, however, as the up and down buttons will not work yet (see Figure 7).

Test the movie to watch the Flash application select the appropriate stream to play

Figure 7. Test the movie to watch the Flash application select the appropriate stream to play

Step 4: Stream switching

The stream switching phase involves swapping between bandwidth streams on demand and continually seeking to match the end user's current position. This operation is generally the same whether switching up to a higher bit rate stream or down to a lower bit rate stream. The purpose of this phase is to cleanly swap between the old and new streams using the available bandwidth with minimal impact to the end user while still maintaining accurate monitoring.

Recommended approach

A clean, efficient procedure for stream switching that does not over-complicate the issue is the pause-and-switch process. Once you have identified the need to swap up or down in bit rate, track the current time in the playing stream, close the existing stream so as not to double up end user bandwidth requirements, display a friendly icon or message that you are swapping (because generally this operation causes a minor delay between swapping), then start the new bit rate stream and immediately make it seek to the previously tracked position to match the old video.

To reduce the delay between swapping tracks it is highly recommended to use a quick play buffer. You can set the initial buffer on the new streaming track very low (approximately one or two seconds for non-HD video, and a bit higher for HD). Once the quick play buffer is full, switch over to a standard play buffer, which can be anywhere from 5–20 seconds or higher. Since the video will not start to play until the initial buffer has filled, this allows for a quick start and, if sufficient bandwidth exists, should not affect long-term playback. A good side note is that Flash Media Server automatically implements a form of buffer overrun. This means even if you set the buffer time to 1 second, the buffer will actually allow for more data to be loaded in if it can. Generally, Flash Media Server will load in around twice the amount you set your buffer to before it caps the bandwidth allocation for the stream.

Stream switching walkthrough

In this walkthough, we will manually swap streams of different bit rates. This part of the article pertains to Step 5 in the flow chart (see Figure 8).

After selecting the initial stream, the next step is to invoke switching to provide the best experience

Figure 8. After selecting the initial stream, the next step is to invoke switching to provide the best experience

Set up the files

Follow the steps below to prepare the files for the stream switching process:

  1. Copy the UpScaleDownScale sample files to an appropriate location on your computer.
  2. Open UpScaleDownScale.as in the com.bitratesample.document package. Update the rtmpPath property to point to a valid FMS server. If you're using Flash Media Development Server 3, just change www.yourfmsserver.com to localhost: 
    public var rtmpPath:String =
"rtmp://www.yourfmsserver.com/VOD/";

Set up listeners for the up and down buttons

In this section we'll add the functionality to make the up and down buttons select the next highest or lowest bit rate stream, respectively:

  1. Locate the _initListeners() method. Un-comment the code that adds the MouseEvent.CLICK event listeners to the up and down buttons: 
    // Event listeners for the up and down buttons
up_mc.addEventListener( MouseEvent.CLICK, _onClickUp );
down_mc.addEventListener( MouseEvent.CLICK, _onClickDown );
  2. Locate the _onClickUp() event handler method. This is where we'll add the logic to swap streams up one bit rate "step."
  3. Under the comment that begins // Make sure we have, add an if...else conditional statement that evaluates if _currentVideoIndex is greater than 0 (zero).
  4. If the condition is true, then decrement _currentVideoIndex by 1 (one), then call the _swapStreams() method, passing it the video object from the _streamList array using the value of _currentVideoIndex: 
    // Make sure we have a new stream to swap with - the array is sorted highest to lowest
if( _currentVideoIndex > 0 )
{
_currentVideoIndex--;
_swapStream( _streamList[ _currentVideoIndex ] );
}

    Note: The _streamList array is sorted highest to lowest. That is the reason that we've decreased the value of _currentVideoIndex to "up scale" the stream.

  5. In the else condition call the log() method and pass it the following message: 

    "*** You are at the highest bit rate! ***".
else
{
log( "*** You are at the highest bit rate! ***" );
}
  6. Locate the _onClickDown() event handler method. In this step we'll add similar logic to swap streams down 1 bit rate "step."
  7. Under the comment that begins // Make sure we have, add an if...else conditional statement that evaluates if the value of _currentVideoIndex is less than _streamListLength minus 1 (one).
  8. If the condition is true, increment the value of _currentVideoIndex by 1 (one), then call the _swapStreams() method, passing it the video object from the _streamList array using the value of _currentVideoIndex. The code should look like this: 

    // Make sure we have a new stream to swap with - the array is sorted highest to lowest
if( _currentVideoIndex < _streamListLength - 1 )
{
_currentVideoIndex++;
_swapStream( _streamList[ _currentVideoIndex ] );
}
  9. In the else condition, call the log() method and pass it the following message: 

    "*** You are at the highest bit rate! ***".
else
{
log( "*** You are at the lowest bit rate! ***" );
}
  10. Locate the _swapStream() method. Under the comment // Pause the current stream, call the pause() method on the _ns NetStream Object, like this: 

    // Pause the current stream
_ns.pause();
  11. Under the comment // Store the current time, create a local variable seekTime (typed as an int) to store the playhead position of the current stream. Set the value of seekTime equal to _ns.time: 
    // Store the current time
var seekTime:Number = _ns.time;
  12. Under the comment // Play the new stream, call the play() method of the _ns NetStream Object and pass it the path property of the p_newStream video object: 

    // Play the new stream
_ns.play( p_newStream.path );
  13. Under the comment // Seek to the saved time in the new stream, call the seek() method of the _ns NetStream object and pass it the local variable seekTime: 

    // Seek to the saved time in the new stream
_ns.seek( seekTime );

  14. Under the comment // Make sure the stream is playing, call the resume() method on the _ns NetStream Object:

    // Make sure the stream is playing
_ns.resume();

The completed _swapStreams() method should look like the following:

private function _swapStream( p_newStream:Object ):void
     {
     log( "Swap the stream with " + p_newStream.path );
   
     // Pause the current stream
     _ns.pause();
   
     // Store the current time
     var seekTime:int = _ns.time;
   
     // Play the new stream
     _ns.play( p_newStream.path ); 
   
     //Seek to the saved time in the new stream
     _ns.seek( seekTime );
   
     //Make sure the stream is playing
     _ns.resume();
     }

Save all of the files. Open UpScaleDownScale.fla and test the movie.

The initial video should play as soon as the application loads. You should now be able to swap between the different bit rate streams using the up and down buttons. The status text field displays the name of each new stream as you swap them (see Figure 9).

Click the up and down buttons to watch the stream swap up and down in bit rate

Figure 9. Click the up and down buttons to watch the stream swap up and down in bit rate

Step 5: Bandwidth monitoring / dynamic stream switching

The bandwidth monitoring phase usually begins as soon as the stream has successfully started and is currently playing. It is a persistent process that should continue until the end of the video is near. Once the video stream is near the end, there's no point in continuing to monitor and switch the stream. Implementing bandwidth monitoring can be the most technically complicated portion of any solution. The main difficulty arises from trying to monitor and determine accurate bandwidth fluctuation while the video stream is running and consuming great quantities of bandwidth itself.

This phase includes two sub-parts: monitoring for a bandwidth deficiency that requires swapping to downgrade the stream; and monitoring for excess bandwidth that allows swapping up to higher stream quality.

Recommended approach

When monitoring the need to step down a stream's bit rate, it is usually most straightforward to check for buffer empty events which indicate that the amount of bandwidth available for the stream is not enough to continuously play back the video without interruption. The preferred solution for this scenario is to set a threshold number of buffer empty events that you feel is acceptable for your users during playback. Once that threshold has been reached or exceeded, perform the downgrade operation. Generally, if a threshold of two or more buffer empty events have triggered within standard playback, it is a good idea to consider swapping to a lower bit rate stream.

Monitoring the need to step up the bit rate of a stream is a more complex operation. There isn't an event that fires an alert when extra bandwidth is available. Instead, additional logic and testing strategies are needed once it appears that there is enough slack in the stream to calculate bandwidth on the fly.

In order to avoid impacting the user's experience, the test to step up to a higher bit rate stream is invoked only after waiting for a defined period of time during playback in which no issues have occurred. Once the standard buffer size is full and there have been no issues, start a timer to check that playback is uninterrupted and smooth for a reasonable duration (approximately 20–30 seconds). If any buffer issues are triggered during this standard extension test, start the count again. If the count is ever completed, you can assume that there might be additional unused bandwidth and the application can then begin the upscale test.

To perform the upscale test, track the time that the test is started, and identify the current buffer length at that time. Then, set the buffer to a larger amount than it currently is (roughly 2–3 times the current value is sufficient). This causes the stream to consume as much bandwidth as is available. Measure the amount of time it takes to fill the new enlarged buffer. Finally, calculate the current available bandwidth from the amount of time it took to fill the defined amount of bandwidth, while considering the current file's bit rate. The mathematical formulas and process for the playback duration test are contained in the upcoming walkthrough exercise.

For the step up operation, the biggest concern is how long it takes to appropriately delay and run the test to determine the need to upscale, as well as the complexity of the upscale testing implementation.

Dynamic stream switching walkthrough

Now that we've covered creating custom clients, checking bandwidth, and switching streams manually, let's pool all these concepts together to create a stream switching application that automatically selects a stream for the user and dynamically changes streams when appropriate (see Figure 10).

Implement the dynamic stream switching application to automatically step up and down in bit rate as the video is streaming

Figure 10. Implement the dynamic stream switching application to automatically step up and down in bit rate as the video is streaming

Set up the files

Follow the steps below to prepare the files for the dynamic stream switching process:

  1. Copy the StreamSwitching sample files to an appropriate location on your computer. I've already provided a lot of the code for this application; most of the development steps were already covered in the previous walkthroughs. To create a stream switching application, you'll need a custom client class with bandwidth detection. For the purposes of this tutorial, we'll use the SimpleClient class located in the com.bitratesample.net package. Its primary method is the onBWDone() method that manages the response from the server and passes the information along.

    We also need to have a document class for the application. We'll use the StreamSwitcher class in the com.bitratesample.document package. This class is the main application class and initializes the stream management, inputs the available video streams, and contains the main listeners for the user interface. In the _onStreamChanged() method, the class displays a message to the user to let them know that the stream is in the process of changing. It also contains an _onBufferFull() method that removes the message once the switch is complete. There are additional methods that update the application's text field with information about the NetStream and its status.

    There are also several event and value object classes that are provided in the sample files folder. However, the main piece of the application that is still missing is a class with the ability to manage our video streams. We need some logic that decides when to switch streams and identifies the next stream to switch to from the set of available streams. We'll build this class in the following walkthrough.

  2. Open StreamManager.as in the com.bitratesample.component package. This file contains the skeleton for the class that will serve as the heart of our stream switching application. In the previous sections of this article we've already dealt with setting up the stream, listening for statuses, and switching between streams, so the class already contains the logic for those operations. It also already contains the getter/setter methods for the class's various properties. The class also includes a series of event dispatches that broadcast the stream's status messages to the application so that the messages can be displayed in an text field. Those lines use the StreamInfoEvent class in the com.bitratesample.events package.

    The upScaleStream() and downScaleStream() methods are very similar to the _onBitrateUp() and _onBitrateDown() methods described in the previous walkthrough. They play either the next highest or next lowest stream like those methods, but they also broadcast events to the larger application in order to report the stream change.

Initialize the stream

Now that we're familiar with the code in the existing files, let's begin filling in the gaps. First, we'll initialize the stream and its listeners. The basics are already provided, but in order to provide automatic switching, we need to monitor each of the different phases of the process the stream is in, as well as the state of the buffer:

  1. Find the constructor in the StreamManager class. The constructor takes in a client object, an FMS application path, a video object, and an array of video data objects, as well as a few more optional parameters. Set the initial streamStatus of the StreamManager to INITIALIZING under the following comment: 
    //Set the status as initializing
streamStatus = INITIALIZING;
  2. Similar to what we did in the previous walkthrough, the StreamManager sets up the connection, detects the bandwidth and starts playing a stream based on the detected bandwidth. However, things are done a little differently in the startStream() method. Go to that method, and under the //Change the status to be quick_play comment, set the streamStatus to QUICK_PLAY.

  3. Also in the startStream() method, under the comment starting with // set the buffer, set bufferTime to _quickPlayBuffer. This will allow the video to start playing quickly due to the low buffer time. The startStream() method should now look like this:

    public function startStream( p_newVideo:VideoStreamVO ):void
{
    //Change the status to be quick_play
    streamStatus = QUICK_PLAY;
   
    trace( "Start the " + p_newVideo.bitrate + " stream." );

    // set the buffer to _quickPlayBuffer for fast start
    bufferTime = _quickPlayBuffer;

    _ns.play( p_newVideo.path );
}

  4. Now that the video is playing, we need to listen for changes in the buffer. We will be manually detecting the buffer full events, due to the dual problems of buffer overflow and the rules for native buffer full events. We'll use the ENTER_FRAME event to check our buffer, so find _onEnterFrame in the code and under the comment beginning with //Monitor the buffer state, add a new if statement. We want to manually fire a buffer full event if streamStatus is not equal to INITIALIZING and _bufferChangedFlag is set to true and if bufferLength is greater than or equal to bufferTime. To manually fire a buffer full event, we can call the _onBufferFull() method:

    //Monitor buffer state and manually broadcast a full event when
buffer fills
if (streamStatus != INITIALIZING && _bufferChangedFlag
&& bufferLength >= bufferTime )
{ 
    _onBufferFull();
}

  5. Still in the _onEnterFrame() method, add another if block under the //Only update if our NetStream exists comment. This is just a simple test to see if the NetStream (_ns) exists. If it does, set the bufferLength equal to the NetStream object's bufferLength and set currentFPS equal to the NetStream's currentFPS. This will keep our stream stats current. The code should look like this:

    // Only update if our NetStream exists
if( _ns )
{
    bufferLength = _ns.bufferLength;
    currentFPS = _ns.currentFPS;
}

  6. In addition to the monitoring performed in the onEnterFrame listener, we need to listen for buffer empty events and buffer full events. Our existing NetStatus handler will handle both events, but we need to add those specific cases into the handler. Find the _onNetStatus() method and add the following three cases to the end of the select statement after the NetStream.Play.Start case, like this:

    case "NetStream.Buffer.Full":
 {
    trace( "The buffer is full
   and the stream will begin playing." );
    _onBufferFull( p_event );
    break;
 }
 case "NetStream.Buffer.Empty":
 {
    trace( "Data is not being
   received quickly enough to fill the buffer." );
    _onBufferEmpty( p_event
   );

    break;
 } 
 case "NetStream.Play.InsufficientBW":
 {
    trace( "Insufficient
   Bandwidth!!" );
    downScaleStream(); // downscale the stream
	
    break;
 }

Downgrade the stream

With our listeners in place, we can now tell when the stream is in trouble. First we'll handle the downgrading process. We've set a threshold for the amount of empty buffer events we'll allow before downgrading, so we'll count the number of empty buffer events and if it exceeds the threshold, we'll downgrade the stream using methods very similar to the manual downgrade process:

  1. Find the _onBufferEmpty() method and under the //Revert to quick play status with a small buffer time comment, set streamStatus to QUICK_PLAY and bufferTime to _quickPlayBuffer. This will help the stream start playing again quickly after its empty buffer event.

  2. Under the //Increase the empty buffer count comment, increment _bufferEmptyCount and create an if statement to check if _bufferEmptyCount is greater than or equal to _bufferEmptyThreshold. If it is, call the downScaleStream() method. The handler should now look like this:

    private function _onBufferEmpty( p_event:NetStatusEvent = null
):void
{
   //If we're in extended play mode, stop timing it


   //Revert to quick play status with a small buffer time
    streamStatus = QUICK_PLAY;
    bufferTime = _quickPlayBuffer;

   //Increase the empty buffer count and see if that puts us over the threshold
   _bufferEmptyCount++;
   if( _bufferEmptyCount >= _bufferEmptyThreshold )
{
    downScaleStream();
}
}

    Note: If you are streaming H.264 content, it is better to increase your starting buffer to a value slightly larger than the 1 second we are using here.

  3. We still need to reset the _bufferEmptyCount after switching down to a lower bit rate. Locate the downScaleStream() method, and after the // reset empty count comment, set _bufferEmptyCount to 0: 
    // reset the empty count
 _bufferEmptyCount = 0;

Decide whether or not to upgrade the stream

The process for upgrading the stream is a bit trickier. If we're not already on the highest stream, we'll have a waiting period to see if the buffer empties. If it doesn't, we'll increase the size of the buffer. If the extended buffer fills, we will measure the time it took for it to fill and use that to calculate the available bandwidth. Then we can determine whether or not to upgrade:

  1. Find the _onBufferFull() method. Under the //Reset the buffer tracking flag comment, add a line setting the _bufferChangedFlag property to false. Because the native buffer full event fires only after a buffer empty or buffer flush event, we will manually monitor the buffer. If the size of the buffer has changed, then we'll monitor it to see if it has filled. Once it fills, we can turn off the buffer monitoring for the time being:

    //Reset the buffer tracking flag
_bufferChangedFlag = false;

  2. We need to write a large switch statement that decides what to do when the buffer fills based on the streamStatus. In the QUICK_PLAY case of the switch statement, set the streamStatus to STANDARD_PLAY and set the bufferTime to _standardPlayBuffer, like this:

    //Move to standard buffer size
case QUICK_PLAY:
{ 
 streamStatus = STANDARD_PLAY;
 bufferTime = _standardPlayBuffer;

 break;
}

  3. Locate the STANDARD_PLAY case block. When the application encounters a full buffer event during standard play, it will check to see if the highest quality video is already playing. If not, the Timer stored as _extendPlayTimer will clear and restart. The Timer measures a given period of time (_extendedPlayDelayCheck), and if it doesn't encounter any empty buffer events in that period of time, it increases the buffer and waits to see if it receives another buffer full event. Here's the code:

    case STANDARD_PLAY:
 {
 if ( _currentVideoIndex != 0 )//if
not the best video already
 {

this.dispatchEvent( new StreamInfoEvent( StreamInfoEvent.STREAM_INFO, "[
Start Extended Delay: " + _extendedPlayDelayCheck + " ]" ) );
 
 //Start the
Timer for the monitoring period

_extendPlayTimer.reset();
 _extendPlayTimer.start();
 }


 break;
 }

  4. We've started the timer, so let's leave the _onBufferFull handler and find the handler for our timer. In our init() method, the event handler for the TIMER event has already been set for you to _onExtendedDelayDone. Find that method. Right now, it is only broadcasting that the delay is finished. We need to update it to store the current buffer length, go into extended play mode, increase the buffer, and track how long it takes this bigger buffer to fill. Under the comment //start upscale check, store the current bufferLength as _startBufferLength. Then set the streamStatus to EXTENDED_PLAY. Next, increase the bufferTime to the current bufferLength, plus our _extendedPlayBuffer. Finally, we need to track the starting time to calculate the measurement of how fast the extended buffer fills. We can access that number using the built-in getTimer() method and store the resulting value as _startTimeCheck:

    //start upscale check
_startBufferLength = bufferLength;
streamStatus = EXTENDED_PLAY;
bufferTime = bufferLength + _extendedPlayBuffer;
_startTimeCheck = getTimer();

  5. At this point, we've established what happens if the playback has no issues during the waiting period. But what if, during the delay before it goes into extended play, the app receives an empty buffer event? We don't want the timer to keep going. Return to the _onBufferEmpty() method. Under the comment //If we're in extended play mode, stop timing it, call the stop() method of _extendPlayTimer:

    //If we're in extended play mode, stop timing it
_extendPlayTimer.stop();

  6. Return to the _onBufferFull() method. If the application has moved into extended play mode and the buffer fills, it means there is a significant amount of bandwidth and it may be possible to upgrade the video stream. Look at the EXTENDED_PLAY case in the _onBufferFull() method. In this case block, set the streamStatus to EXTENDED_CHECK and call the _checkForUpScale() method, like this:

    case EXTENDED_PLAY:
 {
    streamStatus = EXTENDED_CHECK;
    _checkForUpScale();
    break;
 }

Calculating the upgrade bandwidth

The _checkForUpScale() method is the main logic for performing the step up for this application. So far we've been tracking the time that it takes for the extended buffer to fill. We can use that, along with the bit rate of the current video, to calculate the approximate bandwidth available to our user. Based on the results, we can decide which stream to upgrade the user to, if any.

  1. Find the _checkForUpScale() method and under the //Reset the timer comment, call the reset() method of our Timer, _extendPlayTimer.

  2. Under the //Get the bit rate of the current video comment, create a local variable of the int datatype and set it equal to the bit rate property of the video object at the currentVideoIndex. We are using the custom VideoStreamVO class as a value object to store information about the video streams. To make sure that datatype is enforced, we'll also cast the object returned from the streamList array as a VideoStreamVO. Here's the code:

    //Get the bit rate of the current video
var currentFileKbps:int = (streamList[currentVideoIndex] as
VideoStreamVO).bitrate;
  3. We also need to track how long it took the stream to fill the extended buffer. We stored the start time for extended play mode, so now we need to get the current time to compare against the start time. Under the comment beginning with //Track the time, create a new variable called now that is of the type Number and set it equal to the return value of the getTimer() method.
  4. Let's break out some intense math. Under the comment beginning with //Calculate, create a new var called diffBufferTime of the type int and set it equal to _extendedPlayBuffer minus _startBufferLength. This formula will give us the size of the buffer we filled.
  5. On the next line, create a new var called totalTime of the type Number. To get the value for totalTime, we need to subtract _startTimeCheck from now and divide the difference by 1000. This returns the total time in seconds.
  6. Create a new variable with the type Number on the next line called totalTestBits and set it equal to the currentFileKbps times the diffBuffertime. This gives us the approximate number of bytes downloaded in the time it took for the buffer to fill.
  7. On the next line, create one more new variable typed as Number and give it the name targetFileKbps. Set it equal to the totalTestBits divided by the totalTime. This returns the estimate of how much data our user's connection can safely handle.
  8. Add one more line that sets bandwidth equal to targetFileKbps. The method should now look something like this: 
    private function _checkForUpScale():void
{
 //Reset the timer
 _extendPlayTimer.reset();
 
 //Get the bit rate of the current video
 var currentFileKbps:int =
(streamList[currentVideoIndex] as VideoStreamVO).bitrate;
 
 //Track the time we finished our extended buffer
time
 var now:Number = getTimer();
 
 //Calculate bandwidth performance to see if we can
upgrade the stream
 var diffBufferTime:int = _extendedPlayBuffer -
_startBufferLength;
 var totalTime:Number = (now - _startTimeCheck) /
1000;
 var totalTestBits:Number = currentFileKbps *
diffBufferTime;
 var targetFileKbps:Number = totalTestBits /
totalTime;
 
 bandwidth =
targetFileKbps; 
 
 //find the correct video index of highest
available 
 
 //make sure not same stream
}

Selecting the upgrade stream

To finish off this method, we need to determine which video to play and then play it by setting the currentVideoIndex:

  1. Under the comment starting //find the correct video, create a new variable called newIndex and give it a datatype of int. Set it equal to the value returned by the getBandwidthIndex() method when you pass it the parameter bandwidth:

    //find the correct video index of highest available
var newIndex:int = getBandwidthIndex( bandwidth );

  2. The last thing we need to do is make sure that the video we've selected isn't the current stream. After the //make sure not same stream comment, create an if statement that makes sure that newIndex is not equal to currentVideoIndex. If they are not equal, set the currentVideoIndex equal to newIndex, like this:

    //make sure not same stream
if ( newIndex != currentVideoIndex )
{
    currentVideoIndex = newIndex;
}

  3. We still need to add logic that determines what will happen if the currentVideoIndex is equal to newIndex. This means that we have not reached the next bandwidth threshold, but it is still a possibility. In this situation, we should restart the whole process of testing whether or not we should upgrade the stream. Add an else block to our if statement, and then restart the sequence by setting bufferTimer to the _standardPlayBuffer and setting the streamStatus to STANDARD_PLAY:

    //make sure not same stream
 if ( newIndex != currentVideoIndex )
 {
 currentVideoIndex = newIndex;
 }
 else
 {
 bufferTime = _standardPlayBuffer;
 streamStatus = STANDARD_PLAY;
 }

The complete _checkForUpScale() method should like this:

private function _checkForUpScale():void
{
 //Reset the timer
 _extendPlayTimer.reset();
 
 //Get the bit rate of the current video
 var currentFileKbps:int =
(streamList[currentVideoIndex] as VideoStreamVO).bitrate;
 
 //Track the time we finished our extended buffer
time
 var now:Number = getTimer();
 
 //Calculate bandwidth performance to see if we can
upgrade the stream
 var diffBufferTime:int = _extendedPlayBuffer -
_startBufferLength;
 var totalTime:Number = (now - _startTimeCheck) /
1000;
 var totalTestBits:Number = currentFileKbps *
diffBufferTime;
 var targetFileKbps:Number = totalTestBits /
totalTime;
 
 bandwidth =
targetFileKbps; 
 
 //find the correct video index of highest available
 var newIndex:int = getBandwidthIndex( bandwidth );
 
 //make sure not same stream
 if ( newIndex != currentVideoIndex )
 {
 currentVideoIndex =
newIndex; 
 }
 else
 {
 bufferTime = _standardPlayBuffer;
 streamStatus = STANDARD_PLAY;
 } 
}

By setting the currentVideoIndex, it triggers the setter method for that property, which in turn calls the swapStreams() method and does the transition from the current stream to the upgraded one.

Test the movie, and you should see the video play. But how do you test if your dynamic switching is actually working? You can still manually switch streams, but that doesn't answer the question. The answer is to somehow choke your bandwidth intentionally. We'll go over this process in the next section.

Testing strategies

Testing bandwidth switching capabilities can introduce some challenges when testing on a local system. The easiest means of testing is to use the third party NetLimiter throttling software. It allows you to throttle the incoming traffic for individual applications. This software works for local computer testing, local network testing, and remote testing. If you are using an actual version of Flash Media Server 3 on a local network, testing is doable through other means. The testing environment with the most options for simulating choked bandwidth is on a remote server.

Strategy 1: Opening multiple streams

The first strategy involves opening multiple streams to "choke" the pipe. This is a simple strategy and useful for checking the dynamic down scaling of a video. You can do this on a local network connection, as long as your LAN doesn't have insanely fast bandwidth:

  1. Publish the FLA file with an HTML page.
  2. Open the HTML page in multiple browser windows (2–3).
  3. Test the movie in the Flash authoring tool.

Strategy 2: Throttling HTTP calls

This strategy involves forcing your application to connect over HTTP instead of RTMP. Once this is accomplished you can throttle your HTTP connection though a proxy of some sort, such as ServiceCapture or Charles. Unfortunately, this strategy doesn't seem to be viable for local connections, but works well for remote development servers.

  1. Configure your server to connect over HTTP. If you are running a web server on port 80 you'll need to adjust what port the Flash Media Server listens to when it starts up.
  2. Open {FMS Install}/Flash Media Server 3/conf/fms.ini.

  3. Locate the ADAPTOR.HOSTPORT setting and add the new port as comma separated values, like this:

    ADAPTOR.HOSTPORT = :1935,81
  4. Save the file and start or restart Flash Media Server.
  5. Change your connection string from RTMP to RTMPT and add the following (sample) port: 

    rtmpt://www.myfmsserver.com:81/VOD
  6. Now you can use some sort of proxy to throttle your bandwidth between you and your server while running the application in a browser or in the Flash authoring tool.

Strategy 3: Throttling incoming traffic

This strategy is similar to the previous one, but slightly different. Using the third-party NetLimiter software, you can throttle the incoming or outgoing network traffic for an application. Because it is throttling traffic for just the application, you don't need to update the port tunneling to make sure you capture RTMP traffic as HTTP traffic. If you just limit the bandwidth for Flash, you can test your stream switching this way. By changing zones, you can throttle network traffic, Internet traffic, or traffic on your local machine. This approach is the easiest and simplest means of testing stream switching.

Where to go from here

This article has touched upon many aspects of the challenging issue of dynamic bit rate stream switching when faced with adverse bandwidth situations. The solutions provided in this article focus on a simple, direct path that provides high accuracy and reasonable effort to implement. Take time to consider how these strategies would work best for your applications and how they can be modified or extended. The amount of permutations that are possible to better suit your users needs and scenarios is endless. The more you know about your end users and the environments in which they function, the better you can serve them.

Be sure to look into the new dynamic streaming capabilities of Flash Media Server 3.5 with Flash Player 10. The reality of dynamic stream switching is smoother, more accurate, and much easier to implement. To learn more about the new capabilities and features, check out the following articles:

To learn more about working with Flash Media Server, read these general articles:

Also be sure to visit the Flash Media Server Developer Center to get the newest information, sample projects and tutorials.

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

About the author

David Hassoun is the founder of RealEyes Media, LLC, a digital media firm based in Colorado that focuses on interactive motion media and advanced Flash and Flex platform applications. David has always had a passion for motion media, the power of video, and the challenges of usability and interactivity. David is an Adobe Certified Master Instructor, teaches advanced RIA classes at the University of Denver, serves as the Rocky Mountain Adobe User Group Manager, and has taught and developed advanced Flash and Flex application courses. As a consultant or while employed with other firms, he has worked for a wide range of companies such as American Express, Chase Manhattan, Qwest, Boeing, Macromedia, Adobe, US Air Force, Bechtel/Bettis, and many more. David regularly performs advanced code and technical best practices reviews, and has provided directional advice for international industry leaders over the past years—including many technical, courseware, and application reviews as an industry expert.