Requirements

Prerequisite knowledge

A basic understanding of the OSMF framework, including resources, media factories, media elements and traits, and how they work together.

User level

Advanced

The Open Source Media Framework (OSMF) provides a foundation for building media players, including the tools to build a simple player out of the box. However, building your player might require some features that are not provided in OSMF. In this article I show you how to extend OSMF to meet your player requirements.

The following examples take you through the process of creating custom resources, media factories, media elements, and traits to extend the capabilities of OSMF. The final result will be a player capable of compositing pre-roll and main video media elements into a single playable element, allowing the video element to start playing at an arbitrary time. This player will also extend the basic SWFElement media element provided in the OSMF Framework to give it the temporal qualities of a video element so that it can be played, paused, seeked, or otherwise controlled. Finally I show you how to add custom events to your media elements, and capture these events from your player to perform logging or update the UI.

Although I will walk you through a concrete example of an extended OSMF player (see Figure 1), you may want to use any or all of these techniques to achieve your specific goals. The techniques described in this article can be applied to any project.

Extending resources and media factories

I will start this demonstration of extending OSMF with resources and media factories. In a real-world scenario, your player will likely be getting the data about what media to play dynamically. The data itself could be a simple as a URI, but whether the data comes from FlashVars, an API call to your server, or an XML file, the first thing your player will do is parse the data and create a resource from it. The data in this example is hard-coded for the sake of simplicity.

Example of a resource that contains multiple assets and metadata

Resources should contain all the necessary information to play a single video. The OSMF framework provides one resource, URLResource, for loading a single asset from a URL. This resource is good enough for most simple applications, where playback consists of loading and playing a single piece of media, but anything more complex likely requires creating your own resource.

In this example you will build a resource to play a composite video with multiple elements. Specifically, you will build a resource for an element that contains the URLs for both pre-roll and main video assets, as well as additional metadata to control the start time of the main video. The player uses the start-time metadata in order to start playing the main video at an arbitrary time. The resource contains all three pieces of information: the URL of the pre-roll asset, the URL of the main video asset, and a start time for the main video given in seconds.

Code: com/jibjab/osmf/demo/media/MyResource.as

// // Constants // /** * The namespace URL for this resource */ public static const NAMESPACE:String = "http://ns.jibjab.com/demo/my_resource"; /** * Constructor * * @param prerollURL * @param videoURL * @param videoStartTime */ public function MyResource( thePrerollURL:String, theVideoURL:String, theVideoStartTime:Number, theMediaType:String = "flv" ) { _prerollURL = thePrerollURL; _videoURL = theVideoURL; _videoStartTime = theVideoStartTime; // Used to identify the file type of the main video. mediaType = theMediaType // It's good practice to define a namespace for your resources to avoid conflicts addMetadataValue(NAMESPACE, true); super(); }

As with all resources, our new resource extends MediaResourceBase (refer to MyResource.as to see the full code). The resource simply stores the values for the prerollURL, videoURL, and videoStartTime, which are defined at the time the resource is created. The resource also defines a namespace. It is good practice to define a namespace for your resources in case there are naming conflicts with other resources that your player might use. You will also use this namespace later on when you create the media element factory, so that the factory can check to make sure the correct type of resource is being used to create the video element.

Example of a media factory that loads custom resources

Creating a custom resource is only the first step. The second step is creating the actual media element that the player will use to play the video. OSMF provides the MediaFactory class to do this job. A MediaFactory parses a resource (derived from the MediaResourceBase class, as in the previous example) and creates a new MediaElement object from it. The OSMF DefaultMediaFactory is able to parse a standard URLResource and create a VideoElement, SoundElement, ImageElement, or SWFElement object. It does this by looking at the URLResource's file extension and MIME type, if one is provided. In this example, you will create a MediaFactory that parses your new MyResource instance. The Media element created will be a CompositeElement, which I will explain later. For now, take a look at the MyMediaFactory class.

Code: com/jibjab/osmf/demo/media/MyMediaFactory.as

// // Constants // /** * The sendable timeline id */ public static const MY_RESOURCE_ID:String = "com.jibjab.osmf.demo.elements.MyResource"; public function MyMediaFactory() { super(); init(); } private function init():void { // create a new MediaFactoryItem for the resource var myResourceMediaFactoryItem:MediaFactoryItem = new MediaFactoryItem( MY_RESOURCE_ID, function(resource:MediaResourceBase):Boolean { var hasSendableMetadata:Boolean = resource.getMetadataValue( MyResource.NAMESPACE ) as Boolean; return hasSendableMetadata; }, function():MediaElement { return new MyMediaElement(); }); // register the MediaFactoryItem addItem( myResourceMediaFactoryItem ); }

As with all MediaFactory classes, the new media factory extends MediaFactory. On instantiation, the factory class creates and registers a MediaFactoryItem. A MediaFactoryItem is a delegate class that has two functions: it determines whether the factory is capable of creating a media element from a given resource and it provides a function that does so. In this case, the canHandleResourceFunction simply checks for the presence of the MyResource.NAMESPACE metadata to make sure it is a MyResource instance. The mediaElementCreationFunction simply returns an instance of the custom media element, MyMediaElement, which I explain next.

Using composite and proxy elements

In this section you create a custom media element called MyMediaElement. This element is a composite element, meaning that it is made up of multiple child media elements. But let's first see what composite elements are all about and how to use them.

Composite elements

The goal of this demonstration is to create a video element that plays a pre-roll and a main video with an arbitrary start time. To do this, you are going to composite multiple child elements together into a single media element that exhibits the playback behavior you are looking for. OSMF provides a base element called a CompositeElement that lets you combine two or more child elements into a single media element. Extending from the CompositeElement are the ParallelElement and the SerialElement, which play the child elements all at once or one after each other. In this case, extending the ParallelElement is the easiest approach.

The ParallelElement is a composite element that plays all child elements simultaneously. Using a ParallelElement will mean that the pre-roll and the main movie play simultaneously, but this is not exactly the behavior you want. You want to be able to delay the start time of the main video by an arbitrary number of seconds. This is accomplished by wrapping the main video element in another composite element, a SerialElement. To achieve a notion of a start time, you need to play an empty element before you play the main video element. This empty element will have a duration that is equal to the main video's start time, so when the SerialElement is played, the main video element itself starts only after a time equal to the duration of the empty element, which is the start time value provided in the MyResource instance. In OSMF, the empty element is called a DurationElement.

Proxy elements

A DurationElement is a proxy element. Like a composite element, it wraps another media element; unlike a composite element, however, it wraps only a single element and is designed to alter one or more of the element's traits. A DurationElement alters the temporal traits (TimeTrait, SeekTrait, and PlayTrait) of the element it wraps. In cases where there is no element to wrap, as in this case, it becomes an empty element with a duration—just what you need. Now you see that the custom video element is actually composed of several elements wrapped inside one another to give it the complex playback functionality that you want (see Figure 2).

Extending ParallelElement

Taking a look back at the MyMediaElement Class, you can see that the code overrides the resource property setter in order to build the child elements, as explained previously, from the MyResource instance.

Code: com/jibjab/osmf/demo/media/MyMediaElement.as

override public function set resource( value:MediaResourceBase ):void { super.resource = value; var myResource:MyResource = value as MyResource; // Only generate child elements if the resource is a MyResource instance // otherwise, ignore it. if ( myResource == null ) return; // Generate a standard FLV playback element for the pre roll // and add it as a child var prerollElement:LightweightVideoElement = new LightweightVideoElement( new URLResource( myResource.prerollURL ) ); addChild( prerollElement ); // Generate a composite element for the main video which is // comprised as a DurationElement with a duration equal to // the start time, and the actual FLV playback element for // the video var videoElement:SerialElement = new SerialElement(); videoElement.addChild( new DurationElement( myResource.videoStartTime ) ); // Create an instance of our custom TemporalSWFElement. videoElement.addChild( new TemporalSWFElement( new URLResource( myResource.videoURL ) ) ); addChild( videoElement ); // Add event handlers for custom events here videoElement.addEventListener( MetricsEvent.NEW_PLAY, handleNewPlay ); }

The overridden resource setter takes the resource passed to it and checks to make sure it's a MyResource instance. It then creates the pre-roll element, which is just a standard LightweightVideoElement, adds it as a child, and then creates a SerialElement, which plays the main video after a given delay. A new DurationElement with a duration equal to the videoStartTime of the resource is created and added as the first child in the SerialElement. Then the main video element is created and added as the second child of the SerialElement, and the whole SerialElement is added as a child to MyMediaElement.

Extending Media elements and traits

Now that you understand the basics of extending resources and composite media elements, it's time to take the next step and actually create a completely new media element. Unlike building a Composite element, which takes on the traits of its child elements, building a new media element also requires building traits to do the heavy lifting and carry out the the tasks that the element is meant to perform.

In this example you are going to build a temporal SWFElement media element. The Open Source Media Framework provides a basic SWFElement, but it has only two traits: load and display. This means that it essentially acts like an image and has no temporal properties: no play, pause, or seek, and no concept of time. You could wrap a SWFElement in a DurationElement proxy in order to add temporal properties, but this would not really be adequate. The playback time would not be tied in any way to the SWF movie's timeline and you would not have any ability to pause or seek the movie. By creating your own temporal SWFElement, you can give a loaded SWF file all of the capabilities of a traditional video element.

Creating a temporal SWFElement

This temporal SWFElement inherits from the LoadableElementBase class, as all elements containing a LoadTrait should. In the constructor of TemporalSWFElement, make sure to instantiate a new loader if one is not provided. In this case, you will use SWFLoader, which is the same loader used by the default SWFElement and contains all the necessary code for loading a SWF file into the current context. If your SWF file required additional initialization after loading, you would typically write your own loader implementation, but since you aren't doing anything special here, SWFLoader will do.

public function TemporalSWFElement(resource:MediaResourceBase=null, loader:LoaderBase=null) { if (loader == null) { loader = new SWFLoader(); } super(resource, loader); }

The parent LoadableElementBase defines two functions that you need to override to set up the element's traits. Once the traits are defined, they will be relied upon to do all of the work. The first function, createLoadTrait, is called first when the element is told to load and should create the LoadTrait instance for the element. The parent class instantiates a default LoadTrait during this step, but you need to override it to provide your own LoadTrait implementation, LoaderLoadTrait (well, not really your own implementation, but I'll explain that later). LoaderLoadTrait extends LoadTrait and provides access to the Loader instance that is used to load the SWF file as well as providing bytesLoaded and bytesTotal properties that reflect the load progress:

override protected function createLoadTrait(resource:MediaResourceBase, loader:LoaderBase):LoadTrait { return new LoaderLoadTrait(loader, resource); }

LoaderLoadTrait is actually a class provided in OSMF and is used by SWFElement, but it is undocumented—probably because the OSMF authors are not quite sure if it's ready for prime time and may change it in the future. I've used the class in this example for simplicity, but a word of caution is needed here. This example works with the OSMF 1.5 version of this class but you may run into problems if you are using a different version of the framework. When you are creating your own player, it would be best to create your own load trait as well, even if it is a copy of LoaderLoadTrait. That way, your player will be protected from framework changes in the future.

The second overridden function, processReadyState, handles setting up the rest of the traits after the element has finished loading. Usually, traits like time, play, and seek require a reference to the loaded SWF file to do their job, which is the reason why they are instantiated only after a successful load. In this step, you create a DisplayObjectTrait, TimeTrait, PlayTrait, and SeekTrait and add them to the element. References to these traits are saved for use later on:

override protected function processReadyState():void { var loaderLoadTrait:LoaderLoadTrait = getTrait(MediaTraitType.LOAD) as LoaderLoadTrait; loadedSWF = loaderLoadTrait.loader.content as MovieClip; // Calling unload() before this element is fully loaded may cause the load trait // to be null. Do a quick check here. if ( loadedSWF == null ) return; // Add a DiplayObjectTrait and save a reference. addTrait( MediaTraitType.DISPLAY_OBJECT, new DisplayObjectTrait( loadedSWF, loaderLoadTrait.loader.contentLoaderInfo.width, loaderLoadTrait.loader.contentLoaderInfo.height ) ); displayTrait = getTrait(MediaTraitType.DISPLAY_OBJECT ) as DisplayObjectTrait; // Add a TimeTrait and save a reference. addTrait( MediaTraitType.TIME, new TemporalSWFTimeTrait( loadedSWF ) ); timeTrait = getTrait(MediaTraitType.TIME ) as TemporalSWFTimeTrait; // Add a PlayTrait and save a reference. addTrait( MediaTraitType.PLAY, new TemporalSWFPlayTrait( loadedSWF ) ); playTrait = getTrait(MediaTraitType.PLAY) as TemporalSWFPlayTrait; // Add event handlers to the play trait to handle our custom events. playTrait.addEventListener( TemporalSWFPlayTraitEvent.PLAY_FROM_START, handlePlayFromStartEvent ); // Add a PlayTrait and save a reference. addTrait( MediaTraitType.SEEK, new TemporalSWFSeekTrait( loadedSWF, timeTrait ) ); seekTrait = getTrait(MediaTraitType.SEEK) as TemporalSWFSeekTrait; super.processReadyState(); }

The final piece of the TemporalSWFElement is defining an event handler for a custom trait event that I will discuss later.

Creating traits for play, time, and seek

The custom trait, TemporalSWFPlayTrait, created here can play, pause, and stop a loaded SWF file. All play traits need to extend the base PlayTrait class. All extended play traits work by overriding one or both of the functions playStateChangeStart and playStateChangeEnd. These functions are called before and after every change of state. The trait needs a reference to the main MovieClip instance of the loaded SWF file so that it can control the timeline, so a reference is passed in the constructor and saved. You override playStateChangeStart and, inside the function, do the work of actually playing and stopping the timeline. To help you do this, you set up two utility functions: playAllTimelines and pauseAllTimelines. Because the main MovieClip can have nested MovieClips with their own timelines, these recursive functions just loop through all the movies and play or pause their timelines, too. The playStateChangeStart function checks the newPlayState that is passed in and calls either playAllTimelines or pauseAllTimelines. I've added one additional flag to this function that keeps track of whether the movie is being played for the first time and, if so, dispatches a custom event. The standard OSMF traits dispatch their own events, but you can define your own events that can be used by the parent media element to carry out special tasks. I'll explain more about using these events next, but this particular event will be captured by the parent element and used for analytics to track impressions.

The process of extending a time trait is almost identical to extending a play trait; only, in this case, you inherit from TimeTrait and override one or both of the currentTimeChangeStart and currentTimeChangeEnd functions. TimeTrait also lets you react to changes in duration by overriding the durationChangeStart and durationChangeEnd functions, but in this example, you will only be concerned with current time. Like the TemporalSWFPlayTrait, TemporalSWFTimeTrait requires a reference to the main SWF file. In this case, it uses the reference to get the movie's frame rate and total frames in order to calculate the duration in seconds. It also adds an event listener for ENTER_FRAME to the main MovieClip, which updates the current time on each frame. Finally, you override the currentTimeChangeEnd function in order to check to see if the movie has ended and, if so, signal its completion.

Now that you are familiar with extending traits, the seek trait, TemporalSWFSeekTrait, should be a piece of cake. Extend the base SeekTrait and, this time, override one or both of the seekingChangeStart and seekingChangeEnd functions. In this case you extend seekingChangeStart in order to convert the seek time into a frame number in the main SWF MovieClip and advance to that frame.

Creating and capturing custom events

OSMF provides a host of events that your player can hook into to perform tasks like UI updates or analytics tracking. The MediaPlayer instance dispatches some high-level events itself and redispatches default OSMF trait events. But there are certain situations in which you will want to dispatch your own events—especially if you have written custom media element and trait classes.

Adding events to traits and media elements

If you are using the default MediaPlayer class, it won't know how to redispatch your custom events but you can listen to the events dispatched by the media element directly. Remember that the MyMediaElement class you created is just a composite element, but you added an event listener to the TemporalSWFElement to listen for and redispatch custom MetricsEvents. The TemporalSWFElement will be dispatching these events when it receives a trait event from TemporalSWFPlayTrait. I delineated dispatching the event from the play trait, but I have not yet explained what happens when it is captured by the TemporalSWFElement. In this example, the element just dispatches a new MetricsEvent up the chain, but your element may also perform additional functions. Your element could, for example, pause playback and display a warning screen for PG-13 rated content (i.e., not suitable for children). In this case, you would want to show the warning screen only once, so you would not want to add it to the MyMediaElement composite element as a normal child element. This event handler would be a good place to do so because it is only called once, the first time the video plays.

Capturing events in the controller

Taking a look back at the Main.mxml file, you can see where the event handler for the MetricsEvent is handled. As it is, it just traces a message; but here is where you may want to log the event to an analytics service.

Where to go from here

You should now have a pretty good understanding of the various ways you can extend the Open Source Media Framework to build a player that meets your requirements. You've walked through a specific example, but the principles discussed here can be used in a wide variety of applications. You may not need to use all of the techniques described here in your particular project, but being aware of them should make you confident that OSMF can be used to build even the most complicated players and will allow your player to grow and evolve.

To work with more OSMF samples, check out the Mastering OSMF series on the Adobe Developer Connection.

Sample video assets (Preroll_2010_Web.flv and 98feature.swf) copyright © 2011 JibJab Media Inc. All rights reserved. JibJab, Sendables, Starring You, and JokeBox are registered trademarks of JibJab Media Inc.