Dynamic streaming in Flash Media Server 3.5 – Part 2: ActionScript 3.0 dynamic stream API


- Requirements


Prerequisite knowledge

General familiarity with streaming media and programming ActionScript 3.0 in Adobe Flash CS4 Professional or Adobe Flex.


Required products

Flash Media Server (Download trial)


Sample files

Flash Media Server tools


User level

Intermediate

 

 

Dynamic streaming is the process of streaming video to users and dynamically switching to different streams of the same content, but of different quality and size, as needed. The purpose is to provide the users with the best possible viewing experience their bandwidth and local computer hardware (CPU) can support without the need to continually buffer. Another major goal of dynamic streaming is to make this process smooth and seamless to users, so that if it is necessary to up-scale or down-scale the quality of the stream, it is a smooth and nearly unnoticeable switch that doesn't disrupt the continuous playback.

 

This article explores the new DynamicStream and DynamicStreamItem classes in Adobe Flash Media Server 3.5, which provide a quick and easy way to add dynamic streaming capability rapidly to custom SWF-based video player applications. These classes use a more complex underlying system that I covered at a higher level in Part 1: Overview of the new capabilities. For more information on the lower-level capabilities of dynamic streaming, check out ActionScript guide to dynamic streaming by Abhinav Kapoor.

Overview of dynamic streaming ActionScript 3.0 classes

 

Writing a dynamic streaming manager class is a complex task that requires monitoring many of the new Quality of Service (QoS) metrics available in Adobe Flash Player 10 and Adobe AIR 1.5 with Flash Media Server 3.5, and then creating the appropriate algorithms to react and invoke dynamic streaming capabilities. Adobe has done all the heavy lifting and supplied the optional DynamicStream and DynamicStreamItem classes. The purpose of using these classes is not simply for the functionality they provide, but for how they provide such powerful functionality simply.

 

These classes provide a powerful system that is in use and supported by Adobe at no extra cost, and works excellently with Flash CS4 Professional or Flex. Much of the same code and functionality of the DynamicStream and DynamicStreamItem classes is actually used in the Adobe video playback components. The classes provide a simple and direct API to invoke and manage dynamic streaming, including the ability to specify a set of multi-bitrate files for playback, specify an optional starting bit rate, manually switch to different streams/bit rates, and to control some of the basic monitoring parameters that drive the automated dynamic streaming and switching. The limitations of the DynamicStream class derive from a basic level of control; if you need to impose specific rules around the switching based on selected QoS metrics, then the DynamicStream class may not suffice.

 

Figure 1 illustrates the culmination of all the streaming classes managed by the DynamicStream class and its flow from start to finish for dynamic streaming.

Sequence and flow of class invocation for dynamic streaming

Figure 1. Sequence and flow of class invocation for dynamic streaming

 

  1. The DynamicStream class is instantiated and linked with a NetConnection.
  2. The DynamicStreamItem class instance is created.
  3. Streams and their associated bit rates are added to the DynamicStreamItem class via the addStream(name:String, bitRate:Number) method and stored in a streams property of type Array.
  4. The startPlay(param:DynamicStreamItem) method is called on the DynamicStream instance after it has been instantiated and linked with a NetConnection.
  5. If the startRate property exists or a previously acceptable bit rate was automatically set in the SharedObject from previous playing, then the stream with the closest rate less than or equal to that value will be selected to play from the DynamicStreamItem streams Array; otherwise, the DynamicStream class will pull the lowest bit rate in the streams Array as default.
  6. A NetStreamPlayOptions instance will be created from the values in the DynamicStreamItem instance and a transition of NetStreamPlayTransitions.RESET will be used by default for the first stream to play.
  7. The NetStreamPlayOptions instance will be passed as the parameter to the play2( param:NetStreamPlayOptions) method of the superclass NetStream. (The initial video plays now and the below steps loop continually as needed.)
  8. The info property is monitored, providing the NetStreamInfo instance with the details about the quality of service and the user experience, with a focus on the maxBytesPerSecond property for bandwidth and the droppedFrames property for CPU performance.
  9. If the NetStreamInfo data indicates the user is unable to sustain a smooth high-quality playback of the current stream, it will shift up or down to a stream of the appropriate bit rate from the DynamicStreamItem streams Array. The DynamicStream class will use a NetStreamPlayTransitions.SWITCH property for the NetStreamPlayOptions instance that is generated from the DynamicStreamItem data and the specific bit rate stream selected.
  10. The NetStreamPlayOptions instance with the SWITCH transition is then passed to the play2(param:NetStreamPlayOptions) and the video transitions smoothly.

DynamicStream class

 

The DynamicStream class extends the NetStream class and is meant to be used as a substitute for the NetStream class. It therefore provides all the same base functionality as the NetStream class, but now exposes some additional key API features to simply integrate dynamic streaming functionality. The basic usage of the functionality is driven from the startPlay() method of the DynamicStream class. This method receives one parameter of type DynamicStreamItem that contains the list of streams/videos of the same content but different quality and bit rate. The rest is managed internally by the DynamicStream class.

 

Example 1

//Instantiating the DynamicStream Class
var ds:DynamicStream = new DynamicStream( myNetConnection );

 

That means that you have a level of control over how the DynamicStream class will monitor the Quality of Service properties, such as the available bandwidth and CPU performance indicated by the number of dropped frames. Utilizing the droppedFramesLockDelay property of the DynamicStream class will enable you to set the initial delay to ignore dropped frames, so that, with the regular expected dropping of frames as the video first initializes, the server will not invoke a stream switch.

 

There are three key properties for managing the buffer of the video content as well. The DynamicStream class allows you to set the initial buffer (startBufferLength) to a low value for quick video initialization and playback.

  • Generally, the value for the startBufferLength property should be as low as possible to allow a quick video start, but should be long enough to monitor initially and set the baseline bandwidth availability. A value around 1 second is usually a good starting point.
  • The preferredBufferLength property sets the standard playback buffer for after the video has started and filled the startBufferLength to maintain video playback and monitoring.
  • A cutoff buffer length set from the aggressiveModeBufferLength property provides the lowest level the buffer should reach before more aggressively switching the content to a lower bit rate to avoid a rebuffer stage.

 

Example 2

//Instantiating the DynamicStream Class var ds:DynamicStream = new DynamicStream( myNetConnection );
var ds:DynamicStream = new DynamicStream( myNetConnection ); ds.startBufferLength = 1; ds.preferredBufferLength = 10; ds.agressiveModeBufferLength = 5;

 

The rate at which the performance and bandwidth is monitored via the Quality of Service (QoS) information provided from the NetStream can be throttled to help achieve more a fine-tuned level of performance and switching. By default, the information is gathered and analyzed every four seconds. To change this default, use the switchQOSTimerDelay property on the instance of the DynamicStream class.

 

Another feature of the DynamicStream class that may be of use is the ability to set a bandwidth selection limit for each instance of the class. This could enable you to programmatically limit certain users from being able to switch dynamically to the higher bit rates even if their systems could support it. To implement this, call the setBandwidthLimit() method on the DynamicStream instance and pass one parameter: the bandwidth in kbps to which to limit the user. The default value of -1 means there is no limit. This should generally be paired with the ability of Flash Media Server to limit client-to-server connections to a specified bandwidth as well, either from server configuration in the Application.xml file or, preferably, via a NetConnection.call() method.

 

Example 3

var ds:DynamicStream = new DynamicStream( myNetConnection ); //Set DynamicStream bandwidth limit to 2Mbsp ds.setBandwidthLimit(2000 ); //setting both ServerToClient and ClientToServer to 2.5 Mbps myNetConnection.call("setBandwidthLimit", null, 327680, 327680);

 

The DynamicStream class also provides a full set of API methods for controlling stream switching manually within the set of files defined in the DynamicStreamItem set. To use the manual switching methods, be sure to first turn on manual switching mode by calling the manualSwitchMode() method on the DynamicStream instance and pass the Boolean parameter of true to turn on manual switch mode, or false to disable it (disabled by default). The manual switching methods include methods to switch to a specified stream by either name (switchToStreamName(name:String)) or bit rate (switchToStreamRate(rate:int)). The DynamicStream class also provides methods to switch to a stream relative to the current stream. This is useful if the user will have the option to manually increase or decrease the quality of the stream. This is available from the switchUp() and the switchDown() methods of the DynamicStream instance.

 

As far as simply informative details, the DynamicStream class provides a few properties that can be of value. The maxBandwidth property indicates the maximum available bandwidth in Kbps available to the stream. Also, two properties indicate the current playing stream name (currentStreamName) and the current playing bit rate (currentStreamBitRate).

 

Another less-documented integrated feature of the DynamicStream class is that it automatically stores the last used maximum available bit rate in a Flash shared object and uses this value the next time that particular application initializes on the same computer or device. This can provide a great performance benefit and even more enhanced user experience by trying to select the highest quality that the user is believed to be able to sustain based on previous usage. This may not always be accurate due to a number of network and ISP related factors, but it provides a better starting point than nothing and is integrated and working automatically.

 

To start playing content with the DynamicStream class instance call the startPlay(dsi:DynamicStreamItem) method and pass it an instance of the DynamicStreamItem class with the streams and associated bit rates defined in the streams array. The following section describes the DynamicStreamItem in more detail.

 

DynamicStreamItem class

 

The DynamicStreamItem class is a very simple class with the sole purpose of providing information to the DynamicStream class for playback of a set of multi-bitrate–encoded files. The DynamicStreamItem instance is simply a transfer mechanism to provide the set of files to manage for dynamic streaming to the Dynamic Stream class. The basic usage of this class entails creating an instance, and then adding multiple streams of the same content but different bit rates to the instance via the addStream() method. The addStream() method receives two parameters, streamName and bitRate. The streamName parameter contains the stream name passed as a String. If you're using FLV content, the stream name is the filename without the extension. If you're using H.264 content, then a prefix of "mp4:" is required, and the file extension should be left on the stream name. The bitRate parameter is the stream bit rate in Kbps.

 

Example 4

 

var ds:DynamicStream = new DynamicStream( myNetConnection ); var dsi:DynamicStreamItem = new DynamicStreamItem(); dsi.addStream( "MP4:sample1_500kbps.f4v", 500 ); dsi.addStream( "MP4:sample1_700kbps.f4v", 700 ); dsi.addStream( "MP4:sample1_1000kbps.f4v", 1000 ); ds.startPlay( dsi );

Note: For more advanced users, if there is a virtual key or sub-stream directory in use, the appropriate path or key should be specified on the stream name passed to

addStream(), just as you would normallyNetStream.play() command.

 

Example 5

 

dsi.addStream( "MP4:myDirectory/sample1_1000kbps.f4v", 1000 );

 

The second parameter of addStream() sets the bit rate for the specified stream as Number. The value should be in Kbps and be for the stream's total bit rate, including the audio, video, and data bitrates. If this value does not accurately match the bit rate of the specified stream, then accurate selection and the core base functionality of dynamic streaming may not work properly.

All of the streams added via the addStream() method are stored in a streams property as Array. Each time a new stream is added, the streams array is resorted on the bit rates from lowest to highest. Although you don't need to add your streams from lowest to highest in your code, it is a good recommendation for code readability.

 

Any public properties available on the DynamicStreamItem instance will control the whole set of streams that are added from the addStream() method and stored in the streams array. Some of the properties of the DynamicStreamItem class are geared towards the power of selecting start and duration support for the streams.

 

The DynamicStreamItem class has familiar properties such as len, which indicates the duration to play the specified stream set in seconds:

  • The default for len is -1, which indicates the stream set should play until it is complete. A value of 0 for len indicates to play only 1 frame.
  • The start property is for specifying the start time in the stream set. The default is 0, which indicates the beginning of the stream. A value of -1 for the start property of the DynamicStreamItem instance will indicate to play only a live stream with matching stream names.

 

The DynamicStreamItem class has a startRate property, which enables you to set the initial bit rate that the DynamicStream class will select from the DynamicStreamItem set. The default value of the startRate property is –1, which will select the lowest bit-rate file to start, unless a default maximum value has been set in the shared object by the DynamicStream class. Do not initially set the default selected bit rate to the highest bit rate file unless the user base will most likely be able to support it. Generally, dynamic streaming will perform better for quick initial playback by allowing the DynamicStream class to play a lower-quality stream first, then switch up to a higher-quality stream.

 

Follow Walkthrough 1: Overview of capabilities in Part 1 to create DynamicStream and DynamicStreamItem objects, and then use those objects to initialize dynamic streaming and see it in action.

 

Live streaming vs. video on demand

 

Live streaming has certain differences in behavior than video on demand (VOD) or recorded file streaming. Their switching behaviors are also different. To specify live switching with the DynamicStream class, set the DynamicStreamItem.start parameter to –1. The default of –2, or a value equal to or larger than 0, would imply the VOD case and the DynamicStream class would make switching decisions based on the VOD behaviors.

 

Where to go from here

 

The DynamicStream and DynamicStream classes of the ActionScript 3.0 dynamic stream API provide an excellent foundation for any custom SWF-based video player application. The classes offer an immense amount of power with minimal effort or difficulty of use. Although they are by no means required to integrate dynamic streaming into your applications, they provide a robust level of control leveraging the powerful and detailed new Quality of Service (QoS) metrics available in Flash Player 10 and the NetStreamInfo class.

 

If you need to integrate dynamic streaming capabilities quickly and efficiently into your applications, seriously consider using these classes for a quick implementation with code developed and supported by Adobe. As I covered in this article, the ActionScript 3.0 dynamic streaming API classes are a wrapper for all the new powerful lower-level classes pertaining to streaming with Flash Media Server 3.5 and Flash Player 10. For more information on the lower-level capabilities of dynamic streaming, check out ActionScript guide to dynamic streaming by Abhinav Kapoor.

 

This is Part 2 of a three-part series on the new dynamic stream classes. Part 3 covers integrating dynamic streaming functionality into existing video players, and the concerns and benefits thereof.

 

Be sure also to read the other articles that cover dynamic streaming in the Flash Media Server Developer Center.