Delivering video for Flash Player on mobile devices

Adobe Flash Player has revolutionized the way we interact with rich media content on the web. More than 75% of all online video depends on Flash technology. Now that Adobe has brought Flash Player 10.1, and later, to mobile devices, you can ensure that all this great content can reach users across devices by taking note of recommendations from this article.

When thinking about deploying mobile video services leveraging the Adobe Flash Platform, you should consider several factors in providing a great user experience:

• Mobile video encoding guidelines: What is the recommended encoding profile to target Flash Player 10.1 on devices? This section summarizes key encoding profile settings in a convenient table.
• Video content optimizations: What are best-practices to follow when authoring video content to optimize performance? This section offers tips on the discipline necessary for optimal performance within the constrained environment.
• Video player optimizations: What are best-practices to follow when developing your video player to optimize performance? This section covers various detection mechanisms to determine the appropriate video stream, along with video player design tips for optimal performance.

Based on available hardware capabilities in the market and internal profiling on reference devices, Adobe recommends the encoding parameters in Table 1.

If you are currently serving video based on the encoding profiles included in Table 1, you may not need to re-encode. However, if you are planning to re-encode, Adobe recommends hardware-accelerated codecs, such as H.264, given significant advances around compression effectiveness that translate into improved frame rates and reduced battery drain.

In general, we recommend maintaining the true source frame rate (at time of capture) to preserve quality. For video with true source frame rates greater than 24 fps, however, you may consider downsampling the frame rate by an even factor, such as 2—for example, from 30 to 15 fps.

Despite the above recommendation of 24 fps, we have seen videos with a true source frame rate of 30 fps perform well, as long as more conservative encoding settings (such as the "WiFi – On2/Sorenson" bit rate and resolution per Table 1) and the video player optimization guidelines are followed.

Re-encode in multiple streams to provide an optimal user experience across various network conditions. At minimum, Adobe recommends that three streams be made available (again, per Table 1):

• High-quality, over-WiFi, such as the "WiFi – H.264" configuration at 500 kbps
• Medium-quality, over-3G, such as the "3G – All" configuration at 350 kbps
• Low-quality, over-2.5G, such as the "2.5G – All" configuration at 100 kbps

To improve performance, Adobe recommends avoiding or minimizing use of videos that:

• Are rotated, skewed and transformed through a perspective projection
• Are bitmap-cached
• Have filters, such as blur or glow effects, applied
• Are accessed through BitmapData
• Have alpha (_alpha) or color transforms
• Are transformed through Pixel Bender
• Are blended with vector graphics, including other video, above (overlay)or behind (underlay)
• Are placed on half-pixel boundaries
• Are placed within a windowless-mode (where HTML embed parameter wmode is not set to "window") SWF file
• Use Scale-9 grid

Beyond video content optimizations, video player optimizations are equally important when it comes to performance. The following sections offer best practices to keep in mind when designing and developing your video player.

Detection routines

Given the increasingly diverse device landscape, it is critical to adopt more sophisticated methods to determine the specific capabilities of a given device and subsequently deliver the appropriate user interface and video stream. This information is important because the user's device may have varying CPU capability, memory, and input techniques, factors which are critical in ensuring your visitor has the best possible experience. Based on joint research with gskinner.com, a Flash development firm, Adobe recommends that developers use varying levels of detection and assign appropriate weights: Flash Player detection, network bandwidth detection, device detection, and input and screen-orientation detection.

Detecting Flash Player and version information

The industry-standard mechanism for determining the presence and version of Flash Player on the device is by leveraging a third-party, open-source, MIT-licensed JavaScript library called SWFObject. SWFObject is a small JavaScript file and can be implemented at minimum of one line of JavaScript code. You can learn more about implementation or download the library from the SWFObject documentation hosted by Google Code.

Detecting network bandwidth

Unfortunately, there is currently no native mechanism in Flash Player for determining network conditions—a key data point in deciding which content is appropriate for your target device. You can make a best-effort approximation of network bandwidth by measuring the time it takes to download required resources, such as XML configuration files or external SWF files, at least 50 KB in size:

Keep in mind that this is an approximation that varies based on network load, device background processes, and current signal strength.

If you're employing Adobe Flash Media Server (FMS) to deliver video streams, there are built-in Quality of Service APIs that return the user's current bandwidth. For more information, read Script-based bandwidth detection in the Adobe Flash Media Server Developer Guide.

Detecting CPU architecture

In the mobile space, the majority of devices rely on slower, more energy-efficient processors, such as those that are ARM-based. While not the most robust mechanism, a strong indicator of whether the device is a non-desktop device is to perform a check for the processor type:

Detecting operating system (OS)

Another rule of thumb to consider that aids in determining whether the device is a mobile device is the OS, particularly detecting for an OS that is not applicable to desktop computers:

Detecting device-specific information

You can also discover more granular device information from the UserAgent (UA) string, which can be retrieved through JavaScript via ExternalInterface. While this may reveal various device-specific information—including OS version, device name, and browser—this approach is not recommended, as it targets a specific device and fails to give your site the broadest possible reach. Furthermore, the UA string itself is not guaranteed to be consistent or well-formed across platforms and carriers.

Detecting screen resolution, dots per inch (dpi), and diagonal

A pragmatic approach in distinguishing between a mobile and non-mobile device is to refer to the device's Capabilities.screenResolutionX, Capabilities.screenResolutionY, and Capabilities.screenDPI to determine the device's physical screen size. Also note that mobile devices typically have higher DPI than a typical desktop computer:

Detecting screen orientation

Flash Player will dispatch resize events from the Stage when the device is rotated; this may impact how your UI is displayed:

In this handler you can determine if the device is horizontal or vertical:

Detecting input capabilities

Various devices will have different input capabilities, which has direct implications for the user experience. For instance, a UI that supports rollover interactions would not be appropriate for a multi-touch device or one without cursor support. Understanding these capabilities enables you to tailor the UI to support input mechanisms for optimal user interaction:

Based on the above Flash Player, bandwidth, device type, and input detection routines, the appropriate UI and video streams can then be provided to the user.

Generally speaking, if the above detection routines provide a high level of confidence that the requesting device is a mobile device, defaulting to a lower-quality video stream—for example, the "3G – All" configuration as described previously in the Mobile video encoding guidelines section—further ensures that playback occurs with an acceptable load time and frame rate. You may consider providing users the option to explicitly select a higher-quality video through your video player controls.

Inefficient video players are typically plagued by both excessive script execution and excessive extra-video rendering. The following describe a number of video player optimizations that may help to significantly improve performance on the mobile device.

Simplify rendering complexity

As much as possible, reduce the number of objects on the stage. Simplify the shape of essential buttons by using, for instance, non-complex fill styles like simple rectangular solid color shapes rather than gradients or multiple small lines. If shapes such as the playhead are changing shape or position, they should be updated as infrequently as possible—no more than once per second.

Limit use of timer functions and check for duplicates

The goal here is to minimize ActionScript processing—intervals, enterFrame handlers, callback functions registered with NetStream, mouse event handlers, and other such functionality. When registering a timer function, it will be called as often as possible up to the rate requested; potentially, Flash Player may end up spending all of its time executing script as opposed to drawing video frames. Consequently, video frames are dropped to keep up with the audio and only a small fraction of the frames in the video are rendered. This is especially true for timer functions that update UI components and hence trigger redraws, which are expensive operations.

Do not render items that are not currently visible

Flash Player will render objects that are hidden behind other objects. Objects such as the "swirling hourglass" should be destroyed when they are no longer needed. For objects that you intend to reuse, stop the Timeline and set the visible property to false, or move the object off-stage. For objects that you do not intend to reuse, call removeChild() and set object to null.

Minimize bitmap caching for frequently changing objects

Avoid bitmap caching (via the cacheAsBitmap property) frequently updated objects. When bitmap caching is enabled, the bitmap needs to be re-cached, an expensive operation, each time the object is updated. For example, frequently updated text fields should not be bitmap smoothed, as this impacts video playback time.

Avoid transparency

Where possible, avoid transparency or limit transparent objects to smaller areas. For instance, it may be preferable to convert transparent overlay ads to pre- or post-roll ads.

Avoid drawing APIs

Drawing API calls are particularly expensive because they usually create objects, such as the scroll bar, dynamically on the Stage before rendering them via the rasterizer. Furthermore, these objects are destroyed and recreated every time ActionScript is executed. If possible, create these objects statically in authoring time on the Stage; these reside in memory through different timelines.

Improve ActionScript efficiency

Computer science coding optimizations apply to ActionScript; common techniques, such as avoiding large loops, reducing the depth of function calls, and simplifying the design of object-oriented classes, apply. Refer to Optimizing mobile content for the Adobe Flash Platform for ActionScript 3 performance optimization tips.

Flash Player 10.1 and later bring Flash Player to mobile, which now empowers you to deliver the video experience that your users have come to love on yet another screen. While you will be able to leverage many of your assets from existing projects targeting Flash Player, I hope that this article has left you with a better appreciation for the constrained environment that is mobile and the optimizations necessary to facilitate a compelling user experience.

Although much of this article has revolved around performance optimizations, I strongly encourage you to keep an eye on another equally important aspect: experience design. Mobile is a compelling platform from which to engage your users, and developing a tailored user experience that leverages the mobile context in a meaningful way is paramount.

Please consult the following resources for more information:

1. Learn more about Flash Player 10.1 and later.
2. See Flash Player 10.1 demos and watch Flash Player engineers discuss some of the key new features and challenges in bringing the full Flash Player to mobile devices.
3. Consult the ActionScript 3 Developers Guide and ActionScript 3 Reference for the Adobe Flash Platform for information on the syntax and usage of the APIs in Flash Player 10.1 and later.

Learn how to tailor your content to deliver the best possible user experience on mobile devices. Check out the following documents:

• Optimizing performance for the Flash Platform
• Authoring mobile Flash content for multiple screen sizes