Understanding metadata support and NetStream clients

Flash Player 9 Update 3 updates the FLVPlayback component to add support for high-definition H.264 video formats. With metadata—the information grouped together into the MPEG container—you can access and display a wide array of useful and interesting information such as album art, song titles, chapters, and other text and images from both video and audio files. The FLV file format uses a metadata packet to store this information.  Two new NetStream callbacks, onImageData() and onTextData(), have been added to support metadata functionality in non-FLV H.264 files.

This section covers the following topics:

• Getting metadata via onMetaData() callback
• Getting metadata via onImageData() callback
• Getting metadata via onTextData() callback
• Custom callbacks via custom client class
• New NetStream notifications

Getting metadata via onMetaData() callback

When you use the FLVPlayback component, the onMetaData() callback is handled for you, so you can add a listener for the MetadataEvent.METADATA_RECEIVED event.

In the example below, the onMetaData() function listens for the metadata event:

import fl.video.*;
myFLVPlayback.addEventListener(MetadataEvent.METADATA_RECEIVED,
    handleMetadata);
function handleMetadata(e:MetadataEvent):void {
    var data:Object = e.info;
    ...
}

Alternately, you can access the metadata at any time after that event by accessing the metadata property. When parsing an MP4 file, Flash Player generates data messages that return extended metadata information.

The following fields are supported:

• width: Display width in pixels.
• height: Display height in pixels.
• duration: Duration in seconds.
• avcprofile: AVC profile number such as 55, 77, 100 etc.
• avclevel: AVC IDC level number such as 10, 11, 20, 21 etc.
• aacaot: AAC audio object type; 0, 1 or 2 are supported.
• videoframerate: Frame rate of the video in this MP4.
• seekpoints: Array that lists the available keyframes in a file as time stamps in milliseconds. This is optional as the MP4 file might not contain this information. Generally speaking, most MP4 files will include this by default.
• videocodecid: Usually a string such as "avc1" or "VP6F."
• audiocodecid: Usually a string such as ".mp3" or "mp4a."

• progressivedownloadinfo: Object that provides information from the "pdin" atom. This is optional and many files will not have this field.

  Note: Atoms are data "boxes," a hierarchal structure of data objects containing other data tracks for audio, video, text, etc. As a very simple building block in the file container, each atom name has specific characteristics. For example, a "moov" atom is a parent atom with no content except other atoms; "mdat" atoms carry the raw audio/visual stream; the "ilst" atom is the ID3 equivalent Apple iTunes uses to store metadata for song titles and artist name and album, etc.; and the "pdin" atom contains progressive download information.

• trackinfo: Object that provides information on all the tracks in the MP4 file, including their sample description ID.
• tags: Array of key value pairs representing the information present in the "ilst" atom, which is the equivalent of ID3 tags for MP4 files. These tags are mostly used by iTunes. In the example below, the cover artwork is accessed and displayed, if the artwork is available:

public function
    onMetaData(data:Object):void {
   if ( data.tags != undefined
    && data.tags.covr != undefined ) {
   // see if we have cover artwork
      for ( var i:int;
    i<data.tags.covr.length; i++ ) {
         var loader:Loader = new
    Loader();
         loader.loadBytes(data.tags.covr[i]);
         addChild(loader);
      }
   }
}

There is not a fixed, definite list of metadata tags available. Flash Player simply reads key value pairs and in most cases does not look at the actual keys. It is recommended to introspect the onMetaData message using recurseTrace to figure out what it contains. This is also a good strategy for identifying any additional information not listed here, because metadata is not fixed and the list of supported metadata tends to increase over time.

In the example below, recurseTrace is used to find the onMetaData function to explore the contents:

public function onMetaData(info:Object):void {
        trace(info);
        recurseTrace(info, "");
}
 
private function recurseTrace(info:Object, indent:String):void
{
        for (var i:* in info) {
                if (typeof info[i] == "object") {
                       trace(indent + i + ":");
                       recurseTrace(info[i], indent + " ");
               } else {
                       trace(indent + i + " : " + info[i]);
               }
        }

}

Getting metadata via onImageData() callback

The onImageData method is a callback like onMetaData that sends image data as a byte array through an AMF0 data channel. The image data can be in JPEG, PNG, or GIF formats. As the information is a byte array, this functionality is only supported for ActionScript 3.0 client SWFs. In the example below, onImageData is used to access and display the image data:

public function onImageData(imageData:Object):void {
   // this is the track number this sample is associated with
   trace(imageData.trackid);
   var loader:Loader new Loader();
   // imageData.data is a ByteArray object.
   loader.loadBytes(imageData.data);
   addChild(loader);  
}

Note: For FLVPlayback programming, the onImageData() function must be in the context of a custom client.

Getting metadata via onTextData() callback

The onTextData method is a callback like onMetaData that sends text data through an AMF0 data channel. The text data is always in UTF-8 format and can contain additional information about formatting based on the 3GP timed text specification. This functionality is fully supported in ActionScript 2.0 and 3.0 because it does not use a byte array. In the example below, onTextData is used to display the track ID number and corresponding track text data in the Output window:

public function onTextData(textData:Object):void {
   // this is the track number this sample is associated with
   trace(textData.trackid);
   // prints the text, can be a null string which indicates that
   // the old string on this track should be erased
   trace(textData.text);
}

Note: For FLVPlayback programming, the onTextData() function must be in the context of a custom client.

Custom callbacks via custom client class

The FLVPlayback component automatically creates an instance of the class fl.video.VideoPlayerClient, assigns it to the NetStream's client property, and handles all callback messages that come through the NetStream. The default class fl.video.VideoPlayerClient handles only the callbacks onMetaData() and onCuePoint(). Both of these messages generate events (MetadataEvent.METADATE_RECEIVED and MetadataEvent.CUE_POINT) so that you do not need to register a custom class to handle these callbacks.

However, if you do wish to use custom callbacks, you must register a custom client class. You should extend fl.video.VideoPlayerClient to ensure that the onMetaData() and onCuePoint() handling supplied by VideoPlayer and FLVPlayback will continue to work properly.

The requirements for a custom client implementation are as follows:

• The constructor method must take an fl.video.VideoPlayer instance as its only parameter.
• The custom class must include a ready property which should be set to true when all messages expected at the very beginning of the NetStream have been received. This step is necessary because the VideoPlayer class initially hides the video and plays the beginning of the file to access the correct dimensions and duration of the video. It then quickly rewinds the video to the beginning and unhides it. If the rewind occurs before all messages at the very beginning of the file are received by VideoPlayer, there's a chance those messages may never be received.
• It is highly recommended that you declare the custom class as dynamic to avoid encountering runtime errors. Errors may appear if any callbacks are triggered that the class is not set up to handle.
• It is also recommended to extend fl.video.VideoPlayerClient to ensure proper onMetaData() and onCuePoint() handling.

Use the code below to register the custom client class:

import fl.video.*;
VideoPlayer.netStreamClientClass = MyCustomClient;

The netStreamClientClass property can be set to the class itself (as shown in the example above) or it can be set to the string name of the class. Setting it to the string name will not force the inclusion of the class into the SWF, so that will need to be forced in another way. For example, you could declare a variable of that type to ensure the class is included.

If the netStreamClientClass property is set to an invalid value, then a VideoError will be thrown with the error code: NETSTREAM_CLIENT_CLASS_UNSET.

The following extended example shows a sample client class that handles onImageData():

package {
 
   import fl.video.VideoPlayer;
   import fl.video.VideoPlayerClient;
   import flash.display.Loader;
   import flash.display.DisplayObjectContainer;
   import flash.utils.ByteArray;
 
   /**
    * MyVideoClient subclasses VideoPlayerClient, the default
    * VideoPlayer.netStreamClientClass value. This way all
    * the metadata and cue point handling built in the
    * VideoPlayer works properly.
    *
    * The class is declared dynamic so if any other
   * messages are received that we do not support in
   * this class (onTextData(), other custom
   * messages) no runtime errors will occur.
   */
 
dynamic public class ImageDataVideoClient extends VideoPlayerClient
   {
 
   /**
   * This variable is set in onImageData and is used to help
   * determine when the ready property should be true.
   */
 
      protected var gotImageData:Boolean;
 
   /**
   * The constructor must take a VideoPlayer as its
   * only parameter. It needs to pass that argument
   * along to the super constructor.
   */
 
      public function ImageDataVideoClient(vp:VideoPlayer) {
         super(vp);
         gotImageData = false;
      }
 
   /**
   * Handling for onImageData() message
   * Loads the image bytes into a flash.display.Loader
   * and adds the image to the
   */
   
      public function onImageData(info:Object):void {
      // Only handle onImageData() once. Any time we seek to the
      // start of the file, the message will call this function
      // again
         if (gotImageData)
            return;
         var loader:Loader = new Loader();
         loader.loadBytes(info.data);
         var parent:DisplayObjectContainer = _owner.root as DisplayObjectContainer;
         if (parent) {
            parent.addChildAt(loader, 0);
         }
         gotImageData = true;
      }
 
      public function onTextData(info:Object):void {
         trace(info);
         recurseTrace(info, "");
      }
   
      private function recurseTrace(info:Object, indent:String):void
      {
         for (var i:* in info) {
            if (typeof info[i] == "object") {
               trace(indent + i + ":");
               recurseTrace(info[i], indent + "  ");
            } else {
               trace(indent + i + " : " + info[i]);
            }
         }
      }
   
   /**
   * property that specifies whether early messages have been
   * received so it is OK for the player to rewind back to the
   * beginning. If we allow the VideoPlayer to rewind before
   * all messages at the very beginning of the file are received,
   * we may never receive them.
   *
   * The default class, VideoPlayerClient, only requires
   * onMetaData() to be received before rewinding. onImageData()
   * also appears at the beginning of the file, so we might miss
   * that if we do not override this property and include a check
   * for this data.
   */
   
      override public function get ready():Boolean {
         return (super.ready && gotImageData);
      }
 
   }
}

New NetStream notifications

Two new notifications facilitate the implementation of the playback components:

• NetStream.Play.FileStructureInvalid: This event is sent if the player detects an MP4 with an invalid file structure. Flash Player cannot play files that have invalid file structures.
• NetStream.Play.NoSupportedTrackFound: This event is sent if the player does not detect any supported tracks. If there aren't any supported video, audio or data tracks found, Flash Player does not play the file.

These notifications are handled by the FLVPlayback component and come in the onStatus callback of the NetStream, resulting in a state of VideoState.CONNECTION_ERROR. If you write code that handles NetStreams directly, then you may want to handle these notifications as well.

Where to go from here

We hope that this article has provided you with a good overview of the changes to the FLVPlayback component after installing Flash Player 9 Update 3.

For more information about modifying the FLVPlayback component, check out Controlling Flash video with FLVPlayback programming by Dan Carr.

Page 4 of 4

Previous Page