by Greg Hamer
15 March 2010
You should be familiar with ActionScript 3 and video player authoring concepts.
For organizations that are publishing video, audio, or other rich media on the web or to mobile devices, Adobe has a new Open Source Media Framework (OSMF) that can dramatically lower the time, costs, and risks of creating your video and media players. OSMF-based video and media players run in both Adobe Flash Player and Adobe AIR.
OSMF itself is an ActionScript 3 code library employed by developers authoring media players using Adobe Flash CS4 Professional and Adobe Flex Builder 3 (as well as any other tool for authoring ActionScript applications).
This article will introduce you to the Open Source Media Framework, provide background on Adobe's goals in creating OSMF, summarize OSMF's benefits, and in doing so provide an overview of resources available for further exploration of OSMF.
Adobe has demonstrated tremendous success in establishing Flash Player as a ubiquitous runtime for client user interfaces on the Internet, and increasingly so on mobile devices. For partners targeting Flash Player, its ubiquity has been an enormous boon. This is especially true for partners distributing video and other rich media.
At provenWebVideo.com, we specialize in building interactive media players. Given our specialty in this area, we find Adobe's new OSMF initiative to be a clear sign that Adobe remains committed to staying at the forefront of delivering the solutions that its partners can rely on for video delivery on the web. With OSMF, Adobe is also reaffirming its commitment to openness: open source, open standards, open specifications, open innovation, and collaboration with other stakeholders in the industry.
Adobe is actively working with many industry participants to achieve the following goals:
- Improve video experiences
- Extend video and media experiences to mobile devices
- Help customers to simultaneously:
- Shorten development time for media applications
- Improve monetization of content
- Reduce cost of development
In launching OSMF, Adobe has partnered with Akamai, which earlier founded the Open Video Player initiative. With OSMF, Adobe and Akamai are now building a media player for the Flash Platform with capabilities far beyond those initially contemplated for the Open Video Player initiative.
OSMF itself builds on top of Flash Player, which remains the de facto choice for delivery of rich media on the web: video, audio, motion graphics, interactive games, and applications.
OSMF is also a strategic subset of Adobe's far broader Open Screen Project for mobile devices. With announcements about imminent deployments of Flash Player 10.1 on mobile devices, Adobe is increasingly extending the ease of deployment and development of Flash Player to mobile devices as well.
To put OSMF in some context, it is strictly for creating "client" applications that run in Flash Player (applications that are compiled into SWF file bytecode, which at runtime is executed by Flash Player). OSMF has an ActionScript 3 code base. Currently, developing using OSMF at the version 1.0 release requires developers to have proficiency in ActionScript 3 programming. (It is expected that future OSMF releases will aim to include visual authoring components for Flash Professional CS5 designers.)
Whether for large, strategic media player products, or smaller video player projects, OSMF is enabling developers to transform the media player development process by reducing costs and development risks. It achieves this in the following ways, each of which is discussed in more detail in this section:
Open source means much more beyond just freedom from license costs
The first two words in OSMF's name—"open source"—highlight one of the obvious key benefits of OSMF. It is one of a limited number of unique products from Adobe that, from their inception, are open source. For us and other customers, the advantages of this are manifold.
The first advantage is the cost of development. Open source means that the software is free. There are no licensing costs: no license cost to use, none to develop, and none to deploy. Plus, OSMF is free to modify for any business or organization's unique requirements. Also important for the world of open source, Adobe and its partner Akamai have consciously designed OSMF to be free from risks of license litigation, as Adobe and their partners are creating a wholly new code base and contributing it directly to the OSMF project. The OSMF is published under the Mozilla Public License, version 1.1 (MPL). The OSMF's licensing, patent, and redistribution policies are summarized at the Adobe Open Source wiki.
Open source is about an open development process as well as free licensing. Thus, a significant benefit for developers building on top of OSMF is that all underlying source code can be downloaded from a public Subversion repository.
OSMF's source is also downloadable in release builds, which include release notes, documentation updates, and other resources. Current early release cycles are done in development "sprints" and thus are referred to as sprint releases. OSMF's release build downloads are available from the OSMF download page.
OSMF's open development process also results in Adobe and its partners publicly publishing the specifications used by the OSMF development team. For developers building on top of OSMF, these specifications can be invaluable for understanding the logic underlying OSMF. Links to OSMF's specification documents are publicly available from the OSMF Features and Sprecifications page.
Another key advantage is the source of the software. Adobe is a world-class commercial software company. With OSMF, Adobe is committing resources comparable to what it commits to its commercial products. This includes professional development, quality assurance, documentation, and support resources. In using open source software, being able to rely on the quality of the product is critical; and with Adobe's backing, developers can have high confidence in the quality of the product and available support.
There are many examples in the area of support including some listed in the sections that follow. Three initial areas to highlight are Adobe's establishing, supporting, and maintaining a public bug and feature request database and public support forums for OSMF, and Adobe's community engagement through their user group programs.
OSMF is a nexus for support of new media delivery technologies and standards
As delivery of rich media has migrated from broadcast and DVDs to digital delivery via networks, many behind-the-scenes technologies have been developed by Adobe and CDNs like Akamai to ensure satisfactory user experiences, including reliable delivery. Relative to OSMF, these technologies are concentrated in two key areas:
- The Flash Player client where OSMF applications run
- Networked servers from which OSMF applications receive their content
Prior to OSMF, Adobe has provided a prebuilt video player named FLVPlayback (provided in all versions of Flash since Flash 8). Though FLVPlayback has provided a default starting point for a generation of video player developers, its architecture is not open. FLVPlayback continues to be included with Flash CS5 and remains a great starting point for a quick drop-in single video player. However, OSMF now offers an alternative starting point for more complex video and media players, especially supporting the types of features detailed in this article. Going forward, as Adobe develops new media delivery capabilities, Adobe will provide support for prebuilt video player functionality first (and possibly exclusively) through enhancements to the OSMF. These capabilities will include both client-side capabilities that Adobe continuously adds to Flash Player and server-side capabilities including those added to Adobe Flash Media Server.
New media delivery capabilities that OSMF will initially support include the following:
- Quality of Service APIs (aka QoS)
- Digital Rights Management APIs (aka DRM)
- HTTP Streaming (currently code-named Project Zeri)
- Flash Media Server
- Flash Media Manifest files (aka F4M)
- Digital Video Recorder functionality (aka DVR)
OSMF also supports emerging standards that are focused on describing aspects of media delivery. Some of these standards are targeted at content monetization (for example, through advertising) and others at compositions and playlist instructions. OSMF is the first Adobe-supported product to provide direct support for several of these standards, including:
- Video Ad Serving Template (VAST)
- Media Abstract Sequencing Template (MAST)
- Media RSS (MRSS)
- Distribution Format Exchange Profile (DFXP)
- Synchronized Multimedia Integration Language (SMIL)
From a developer's perspective, OSMF's support for all of these standards includes parser libraries that assist in rapid development of applications consuming any of these file types.
Note this important distinction between OSMF and FLVPlayback. The FLVPlayback architecture is centered on user interface design. It provides prebuilt user interface controls (buttons, progress bars, and others), for which designers can easily change the appearance (for instance, via styling and skinning). The initial release of OSMF, by contrast, includes only default "chrome"; that is, the initial release of OSMF is targeted at development teams seeking full control of their user interface design. As FLVPlayback does now, future OSMF releases from Adobe are anticipated to include prebuilt user interface controls whose appearance can easily be changed via styling and skinning.
OSMF is powered by a modular, extensible plug-in architecture
Potentially, one of OSMF's most important features is its modular, extensible plug-in architecture. The classes included in OSMF provide the core capabilities of media delivery, plus extensibility through plug-ins. Extensibility through plug-ins is essential for customization and integration with common backend services. Any developers in the OSMF community can write plug-ins, including:
- Media player developers
- Service providers who can provide plug-ins directly to their customers (who include media player developers)
- Any developers who can make their plug-ins generally available to the entire OSMF community
In the broadest sense, the OSMF plug-in architecture offers the following advantages:
Minimizing player file size. It is paramount when deploying media players on the web and mobile to keep the downloaded player's file size to a minimum. When targeting Flash Player, this is the size of the compiled SWF file. The OSMF plug-in architecture enables developers to minimize the final compiled SWF file size by including only code that is in each custom player implementation.
Proprietary customization. A list of OSMF partners currently committed to providing OSMF plug-ins is maintained at the OSMF community portal. These partners are currently developing the following types of proprietary plug-ins:
- Advertising delivery
- Content delivery
Open source customization. Examples of plug-in libraries currently in OSMF include the following:
- Captioning support using Distribution Format Exchange Profile (DFXP)
- Playlist support using Media RSS (MRSS)
- Multiple bit rate support using SMIL
- Advertising using VAST and MAST
- User interface component example via a plug-in (see the OSMF ControlBarPluginSample)
Dynamic runtime plug-in loading. This enables loading functionality at runtime, either from the host domain of the media player or from partner domains (CDNs, analytics vendors, and more). When media player developers choose to implement runtime plug-in loading, they enable the plug-in's publisher to push new versions, such as for upgrades.
Note: The OSMF dynamic plug-in loading is secure. OSMF security permits no unintended plug-ins to be loaded. Its dynamic plug-in loading can be enabled exclusively by each custom OSMF media player developer hard-coding the necessary logic. Thus, no unknown plug-ins can be loaded at runtime. For instance, for media player users, there is no interface available for users to source, install, or load plug-ins.
Static compile-time plug-in integration. This enables modularized code to be compiled into the media player. Plug-ins may be provided by third parties. Integrating static plug-ins at compile time offers the following benefits:
- It provides the media player developer complete control over plug-in versioning
- It eliminates potential time delay at run time in loading a plug-in after media player startup
- At runtime, it ensure that the plug-in code is in the media player at startup
The OSMF plug-in architecture is another important distinction between the OSMF and the FLVPlayback component. FLVPlayback, though skinnable, is essentially a closed architecture. Thus, FLVPlayback supports neither standardization nor the easy development of functionality that is available with OSMF via its plug-in capabilities—functionality such as ad serving, analytics, CDN support, and more.
In summary, the OSMF standardized plug-in architecture is a win-win for all players in the OSMF ecosystem: it's a win for media player developers because it reduces the complexity of integration required before OSMF, and it's a win for plug-in providers because it reduces custom integration with individual customers.
OSMF introduces new classes and new abstractions in thinking about media
For experienced ActionScript media player developers, OSMF introduces new classes and conventions into coding media player logic. Rather than coding directly to the Flash Player native media APIs, OSMF developers code to the OSMF classes (which internally make calls to the Flash Player native media APIs). So OSMF development does not require directly instantiating NetStream objects, for example.
OSMF introduces a new universe of classes and concepts. In transitioning to developing media players based on OSMF, developers need to come up to speed on the OSMF way of thinking about media. The remainder of this section provides a distilled sampling of OSMF classes and the concepts resulting from abstractions underpinning OSMF. For greater depth on topics related to developing media players with OSMF, please see the official product documentation:
A good first example of what is new in media player development using the OSMF is that it uses an abstracted base class, named MediaElement, to represent every type of media in the framework. For expressing rich media in OSMF, common descendant classes of MediaElement are the self-evidently named VideoElement, AudioElement, SWFElement, and ImageElement.
OSMF also supports complex media compositions through another descendant of MediaElement, the CompositeElement class. Descended from CompositeElement are the primary classes used in creating compositions: SerialElement and ParallelElement. Figure 1 illustrates a nested combination of compositions with sample code. The illustration shows a top-level ParallelElement (2) with a nested SerialElement (1).
OSMF includes a layout API. For instance, when playing ParallelElement compositions, multiple visible elements can be displayed simultaneously. The OSMF layout API uses facet classes such as RelativeLayoutFacet (which might be used for the ParallelElement illustration in Figure 1) or AbsoluteLayoutFacet (for overlays). In these cases, the OSMF layout facet classes enable positioning the sub-elements in ParallelElement compositions (where multiple subelements are viewable at the same time).
OSMF uses the concept of facet classes beyond just layout. All facet classes descend from the base class Facet. Broadly speaking, facet classes hold metadata relating to individual pieces of media. Examples include (but are not limited to) cue point metadata, Quality of Service (QoS) metadata, etc. The related event model includes classes MetadataEvent, FacetValueEvent, and FacetValueChangeEvent.
In abstracting what is a MediaElement, OSMF also introduces the concept of trait classes. In the abstracted logic of OSMF, trait classes define the capabilities of each MediaElement. A good summary of standard trait classes in OSMF can be found by reviewing the constants on the OSMF class MediaTraitType. For instance, both classes VideoElement and AudioElement have the traits LoadTrait, PlayTrait, and SeekTrait. However, since instances of AudioElement expose no DisplayObject, AudioElement does not have a ViewTrait, whereas VideoElement does.
To support building analytics into media players, OSMF also includes MediaElement descendants such as BeaconElement.
OSMF includes a number of runnable OSMF examples that can be executed through an OSMF-based example player (see Figure 3). The example player's source code is included in the OSMF Subversion repository. A compiled version of the example player is runnable online.
The example player loads a variety of OSMF demonstration examples (including several that illustrate error conditions such as load failures and invalid media URLs). Each demonstration example can by run by selecting it from the player's list of examples along the left. A brief description of each example appears in the upper-right corner after you select each one.
The OSMF example player provides a quick way to explore OSMF's general capabilities.
Where to go from here
With OSMF, Adobe is transforming the way that media and video player development is done. By creating a powerful open-source media player development framework, developers targeting Flash Player can now create video players more quickly and cost effectively with more capabilities and integrate them with more services than ever before.
If you are a media player developer, your next step from here is to dive in. Download from the OSMF downloads page—to get the most recent OSMF sprint release—both the source ZIP and the release notes PDF. To get your development environment set up, follow the instructions in the release notes under the "Getting Started Instructions" section heading. These instructions provide the necessary details on using OSMF in Adobe Flex Builder 3 and/or using OSMF in Flash CS4.
To get an in-depth understanding of the new classes and conventions that OSMF introduces, see both the OSMF Developer's Guide and ActionScript 3 Language Reference, including the Open Source Media Framework API.