Optimizing performance of applications for connected TVs

Ensemble Systems has partnered with Adobe to develop movie viewing applications for Samsung's 2011 models of Smart TVs and Blu-ray players for EPIX and CNET. Developing the viewer apps for these devices posed some interesting challenges because the Samsung Smart TV and Blue-ray player devices are constrained in both processing power and available memory. The players we developed present selections of video clips and movie titles from an online catalog. Users select a show from the menu, which plays back on their TV or streams over the Internet on demand (see Figure 1).

These applications were demonstrated at the 2011 International Consumer Electronics Show, running on the 2010 Samsung Blu-ray player. Adobe AIR will be embedded in the 2011 models of Samsung Smart TVs and Blu-ray players.

Generally speaking, the processors in connected TVs and Blu-ray players today are not as powerful as desktop computers—but they do have powerful 2D graphics hardware acceleration and capable video decoders that require fewer system resources. During the process of building these projects, our development team at Ensemble learned valuable lessons and best practices to deliver a good user experience on TV and Blu-ray player hardware.

In this article, I'll describe some of these considerations and highlight the lessons we learned to optimize performance of rich Internet applications for Smart TV and Blu-ray players. Although the applications are conceptually simple and reasonably straightforward to develop in ActionScript, we found that it was necessary to pay close attention to the details to ensure good performance. It's particularly important to be careful when dealing with project assets to achieve smooth movement in animations and transitions.

Both the EPIX and CNET applications present movie catalogs with a hierarchy of menus. The navigation elements display appealing animations as the user transitions through the various levels of content. As the user selects a movie, the player loads more details about their selection; in other cases, it also loads the movie player. Movie playback is required to play the video smoothly. The player includes the standard playback controls, such as pause, fast-forward, and other related buttons.

Current hardware is constrained both in terms of memory and vector drawing ability. We couldn't use the classic trick of preparing and caching bitmaps in advance because we found that performance suffered due to processing the initial drawing; memory limitations were also apparent when the app attempted to store too much image data at once. We realized that the key involved drawing bitmaps as efficiently as possible and displaying the graphics needed only for that moment—nothing more. Additionally, since the memory limitations restricted our ability to cache image data, we focused our efforts on recreating the screens more efficiently. These strategies really improved overall performance.

In this section, I'll outline the changes we made to enable the devices to display the graphic content more efficiently.

Caching bitmaps

When developing in ActionScript, it is a common practice to use the cacheAsBitmap property on DisplayObject . We certainly made heavy use of it, but cacheAsBitmap only works on the display object itself, not its children. When developing these projects, we found it useful to render complex but static displays to a bitmap and then display the bitmap instead.

The trick here involves generating the bitmaps we need for the current display and then throwing them away as soon as they are no longer visible on the screen.

Vector objects

It is helpful to use the Graphics methods with vector drawing objects to precisely control a display's appearance. However, be aware that they perform slowly when used in conjunction with scrolling content. When dealing with scrolling elements, we were forced to use cacheAsBitmap . This approach works reasonably well. When absolutely necessary, you can render off-screen bitmaps of complex content, as mentioned previously.

Efficient bitmaps

After experimenting, we discovered that the direct use of bitmaps as bitmap fills on sprites or shapes performed better than the native flash.display.Bitmap object. This strategy appears to use less memory overhead. Furthermore, when you use sprites, you gain the capability for the bitmap to have children, which is convenient when displaying captions.

We ended up writing our own image class that behaved like the standard bitmap class (with a source property that loads from a URL) and drew more efficiently.

Scaling and smoothing bitmaps

The view apps we developed required the ability to scroll large quantities of movie thumbnail bitmaps smoothly on the screen. In many cases, the bitmaps came from the server in a larger size than the thumbnails to be displayed. We found that if you simply scale the bitmaps down to fit, scrolling performance is impacted. It appears that the original bitmap data is used during scrolling, instead of the scaled displayed bitmap data. Similarly, the smoothed bitmaps are slower compared to non-smoothed bitmaps when scrolling.

To resolve this, we generated the bitmap images used for the movie thumbnails as sprites and applied the bitmap fills with full scaling and smoothing. Then we explicitly rendered the resulting image sprite to a new bitmap object that was non-transparent, non-smoothed, and unscaled in order to capture the displayed result for scrolling purposes.

Our displayed movie thumbnails consisted of these final efficient bitmaps, which scrolled much more smoothly.

Transparency

The scrolling performance is also affected significantly when transparency controls are set. If you don't need transparency, ensure that your bitmaps are drawn with transparency set to false . It's a best practice to set the opaqueBackground property to the desired background color instead.

The following tips highlight other helpful strategies we followed in order to improve the performance of the viewer apps and make them play faster in devices.

Font rendering

To achieve efficient font rendering, use embedded fonts. Although you certainly gain better display control with embedded compared to system fonts, the drawing performance of embedded fonts is the primary advantage needed when building these types of applications.

Another consideration is to set the antiAliasType property of text fields to NORMAL or ADVANCED , as needed. While advanced anti-aliasing provides the best visual results, the normal setting draws faster. Our team generally used the advanced setting for static text and applied normal anti-aliasing to moving text.

Also, setting cacheAsBitmap to true helps significantly when text is included in scrolled content.

Content layering

It's very important to minimize the amount of content on the screen at any one time. For example, in many projects it is acceptable to overlay temporary screens on a previous screen. However, we found in constrained devices (like connected TVs) that underlying content can impact performance even if the new screen content is non-transparent and completely covering it. Essentially, the non-visible content still factors into clipping and drawing calculations.

We first encountered this issue when we overlaid a full-screen video player for movie playback over the top of the previous movie selection screen. The application suffered from poor video playback with stuttering. But after adding code to ensure that all underlying content was removed, the video playback performance improved.

Efficient layouts

Component libraries like Flex greatly aid in the efficient development of components and component placement. They provide controls that allow layouts to be implicitly determined (using features such as percentage-based sizing). However, these abilities come with a runtime cost: it takes more time for components to "settle" in and be displayed as they are measured and sized in layout loops.

When developing these viewer apps, we used pure ActionScript-only components with little to no implicit placement. In our projects, the apps were simple enough that placement could be either constant positions or readily computed from the size of the parent component— using code such as this:

This approach greatly improved app performance, allowing components to be placed on the Stage much more immediately, and resulted in smoother transitions.

As hardware improves, this issue will matter less. In the future, you'll have more processing power in devices that can leverage the improvements in development effort that Flex components provide. However, in the short term, as you develop apps for performance-constrained devices, it's important to be conscious of the runtime impact of the layout model you use and look for alternatives.

Many existing component libraries are available to facilitate efficient development of components and component placement. Flex is particularly powerful and useful when building applications in general. Note that the latest version of Flash Builder 4.5 has greatly improved support for constrained devices, especially mobile devices, and the layout performance issue has been significantly addressed.

Tweened animations

A good tweening library is a very common tool in every Flash developer's arsenal. On high-performing systems, your choice really depends on stylistic preferences and the features available in the API. However, when developing for poor-performing devices, the tweening library's implementation can make a big difference in consistent performance, predictable execution times, and the occurance of dropped frames.

After testing, we found the GreenSock tween library is quite good in terms of runtime efficiency and the ability to execute tweens with low overhead.

In other situations, we developed explicit frame event handlers and raw timers directly. When you use this approach, take care to avoid using code that counts frames. Always measure the elapsed time instead.

In general, it is a best practice to write code with few (if any) frame event handlers. It is optimal if the code is independent of frame rate. In our projects, we used the default frame rate of 24. We discovered that increasing the frame rate simply increases the per-second amount of processing by the runtime; decreasing it tends to make animations choppier.

Smooth scrolling

In these projects, we originally attempted to scroll entire collections of movie thumbnails together by modifying their individual y properties. This is inefficient. It is much more efficient to place all scrollable content in a single parent container and then scroll that parent container in a single transform.

Performance optimization can be tricky and time-consuming. Strive to follow general best practices in advance, but realize that ultimately every project requires a lot of trial and error. Sometimes seemingly unrelated changes made when bug fixing can undo achieved performance gains, so it is critical to continually test your project (early and often) throughout the entire development and deploy phases.

To learn more about developing for connected TVs, visit the Flash Platform for TV technology center. To get more information about building RIAs, check out the Rich Internet application development center.