This article outlines Adobe's recommendations for developing and enhancing the performance of dynamic streaming with Adobe Flash Media Server 3.5 and Adobe Flash Player 10 for streaming recorded media on demand.
Encoding recommendations for multiple bit-rate delivery
If your target viewer covers a broad spectrum of bandwidth capabilities, it is best to keep a wide range of stream bit rate encodings while keeping a large enough difference between successive bit rates. Too many bit rates too close to one another could result in too many stream switches, even with smaller bandwidth fluctuations. Besides the slight overhead in switching, the viewer's experience with too-frequent quality fluctuations may not be pleasant. Meanwhile, too few streams with a huge gap in bit rates would not provide the best quality or the most optimal stream for a particular bandwidth.
Adobe recommends using the bit rates given in Table 1 for dynamic streaming on demand. If the intended target users are towards the higher end of the bandwidth spectrum with at least a DSL connection, then the first couple of bit rates could be skipped. The frame rate for videos below a bit rate of 100 Kbps could be set to lower values such as 15 fps, but at bit rates higher than 300 Kbps, a frame rate of at least 25 fps and ideally 30 fps is recommended.
Table 1. Recommended bit rates for dynamic streaming on demand
* Note: Based on the IDC 2008 Consumer Panel Broadband Survey. Each figure represents the percentage of users who have the bandwidth to support the respective total bit rate in that category. For example, 25% of users have bandwidth of at least 1200 Kbps to support the D1 video type but don't have the higher bandwidth needed to support the next higher bit rate of 1800 Kbps.
Achieving the smoothest transition between streams
There are no specific encoding requirements for media to support dynamic streaming. Existing media files that you have today (in FLV, F4V, or MP4 formats) are compatible and will produce a reasonable transition between streams. The following recommendations will help create the smoothest experience during transitions.
Keep the audio bit rates and sampling rates the same to provide a seamless switch between them as the stream plays. Switching between two streams with the same sampling and bit rates will allow seamless transition. Switching between incompatible bit rates or sampling rates is possible, but could result in a slight audible "pop" sound at the time of the transition.
For lower bit rate streams, encoding the channel as mono and maintaining the same bit rate (that is half the bit rate of their stereo equivalents) will provide additional optimizations that enhance seamless transitions to their stereo equivalents.
For higher bit rates, the video encoding frame size (video resolution) should not be set higher than the source capture size. Doing do would end up scaling the video and not necessarily produce a higher quality image. Keeping the maximum encoding size limited to the maximum source video size gives the best quality at a particular bit rate.
The video frame size should not exceed the maximum video frame size on the subscriber's end. If the client's video frame is restricted to a particular size, encoding at a higher size will not add any value, and the bits may be better conserved for encoding quality than size.
Keeping a constant video keyframe interval instead of a variable keyframe interval ensures that the keyframes in the streams are not very far away from one another. Since the server switches streams on a video keyframe, having them too far away could cause a delay in the switch to happen on the server side.
Smooth video switching
Keep the timeline of the multiple encoded files identical. If the streams have different start points or are not exactly the same pieces of content, then there could be a perceivable jump in the playback during a transition.
For a faster switch with optimal keyframe interval and client-side buffer, when Flash Media Server (FMS) receives a "switch" command, the server waits for a keyframe to switch to the new stream. FMS looks for the keyframes in the new stream in chunks equal to the client's buffer size (NetStream.bufferTime), so having a client buffer larger than the keyframe interval of the stream would help with a fast switch response time from the server—in other words, a shorter delay between a "switch" call to the server and the client receiving bits from the new stream in response.
The shorter the keyframe interval, the faster the switch will be. However, keeping the keyframes too close to one another will increase the file size and hence lower quality at a specific bit rate, so the following values are considered most optimal:
- Keyframe interval: 5 sec.
- Client-side buffer: 6–10 sec.
Achieving faster switch response using bandwidth throttling
Flash Media Server starts with a very high default value for ServerToClient bandwidth in Application.xml, and hence assumes a high client bandwidth availability and tries to push a lot of data quickly.
If the client's bandwidth drops and the player application requests that the server switch to a lower bit rate, the server could take a lot of time to push the already queued-up bits at the previously set, higher bit-rate stream before it could switch to the lower bit-rate version. It is useful to keep the server's outgoing buffer small by keeping the ServerToClient bandwidth limit smaller. This would allow the server to push the smaller queue of outgoing data faster to the client and switch to the lower bit-rate stream more quickly. Keeping the value too low, however, would limit the server's ability to push more data when the client's bandwidth increases again.
Ideally, this bandwidth limit should be set to a value slightly above the maximum bit rate of the streams being sent. For example, if a content has streams of 500 Kbps, 800 Kbps, 1.2 Mbps, 1.8 Mbps, and 2.4 Mbps, then setting the ServerToClient bandwidth limit to 2.5 Mbps or 327,680 bps would provide the maximum stability. This can either be set in Application.xml at the server or, better yet, set on the NetConnection object from the client-side application, as the following examples show.
Application.xml found in the app directory:
<!-- Specified in bytes/sec -->
Alternatively, in the client application, you can use the following ActionScript to set the bandwidth limit from the SWF file manually.
Use client bandwidth detection with NetStreamInfo.maxBytesPerSecond instead of using the server's native bandwidth detection calls:
nc.call("setBandwidthLimit", null, 327680, 327680);
//setting both ServerToClient and ClientToServer to 2.5 Mbps, though ClientToServer doesn't matter here
Server-side bandwith detection has a startup overhead and can only provide the bandwidth information at the start of the session, while the client's maxBytesPerSecond is dynamic and reflects the bandwidth capacity at the time.
QoS properties to consider for smooth stream switching
While NetStreamInfo.currentBytesPerSecond provides the true incoming data rate of the stream, it may not be the right value to gauge the bandwidth capacity of the client. The incoming data rate tends to fluctuate because Flash Media Server sends data in bursts for server-side scalability reasons. It will push enough data to fill the client-side buffer up to twice the requested buffer length (NetStream.bufferTime) and then stop; so the currentBytesPerSecond will start showing 0 during that situation.
NetStreamInfo.maxBytesPerSecond takes server-side throttling into consideration and hence will always provide the maximum capacity of the client's network to which the stream has access. It provides a good value to switch a stream up to the best bit rate the client can handle in terms of network resources and give the viewer the best possible quality. This property requires FMS 3.5 or later.
Another property to consider is NetStreamInfo.droppedFrames, which provides the absolute number of video frames dropped due to limited CPU resources on the client's machine. If the bit rate or the resolution of the video is much higher than the client CPU can handle, the video will tend to stutter and drop video frames in order to keep the synch going with the audio. If the video is dropping a lot of frames, then that should be a good trigger to switch to a lower bit rate version of the content to provide a smoother experience. Video smoothing also has an impact on CPU usage, and if the video is dropping a lot of frames, it could be turned off prior to switching down to a lower bitrate in order to see if smoothing alone brings down the dropped frame count.
NetStream.bufferLength is also a critical property to monitor to make switch-down decisions. Besides NetStream.info.maxBytesPerSecond, NetStream.bufferLength will also start dropping when the client's bandwidth drops and is not able to sustain the buffer length with the current bit rate of the video. If the buffer length drops below a certain threshold, it should trigger a switch down to a lower bit rate that could be calculated with the NetStream.info.maxBytesPerSecond property.
When not to initiate a stream switch
As a best practice, it is recommended that a new switch not be initiated in the following scenarios:
- While the stream is buffering. This means following a Buffer.Empty event until the Buffer.Full event is received or until the first Buffer.Full event on a new stream.
- While the server is processing a previous switch call. Wait until the client receives the NetStream.Play.Transition event for the prior switch before making another call.
The H.264 codec can provide some useful additional settings to optimize video quality while keeping the bandwidth low.
Use of B-frames: Besides keyframes and predicted intermediate (P) frames, H.264 also supports B-frames. Unlike P-frames, which only look backwards at keyframes and other P-frames as a reference to encode a current frame, B-frames are bidirectional and look at both forward and backward frames as reference and hence are more efficient.
B-frames provide quality enhancements because they provide greater image detail with very good compression efficiency. The trade off, however, is slightly more decoding requirements. There is a 5–10% increase in CPU usage, and B-frames are not supported with the Baseline profile. If the playback CPU requirements are not very low, use of B-frames with an interval value of 2 or 3 is recommended.
Entropy encoding: Using CABAC (context-adaptive binary arithmetic coding) provides better quality than CAVLC (context-adaptive variable-length coding), but at a cost of approximately 3–5% of CPU usage. Again, if lower-end system configurations are not targeted, use of CABAC is suggested. Since CABAC is also not available with the Baseline profile, use of Main profile or higher is recommended.
Flash Media Encoding Server and Sorenson Squeeze provide the configurations given here. Flash Media Live Encoder and Adobe Media Encoder do not expose B-frame and entropy encoding settings at this time.
Where to go from here
For more information, please refer to these other articles on dynamic streaming:
Tutorials & Samples