Open Source Media Framework (OSMF) 1.6 (and more recent versions) supports the inclusion of one or more alternative audio tracks with a single HTTP video stream. This practice, known as late-binding audio, enables content providers to deliver video with any number of alternate language tracks without having to duplicate and repackage the video for each audio track. Users can switch between the audio tracks while maintaining full synchronization between audio and video. OSMF detects the presence of the alternate audio from a Flash Media Manifest (.f4m) file, which has been modified to include bootstrapping information and other metadata about the alternate audio tracks.

This article guides you through the process of delivering an alternate language audio track to an on-demand video file (VOD) streamed over HTTP using the late-binding audio feature. Before beginning, make sure you understand the HTTP Dynamic Streaming (HDS) workflow. You may want to refer to the following HDS resources:

Required software and resources

To complete the steps outlined in this article, you will need the following:

Flash Player 10.1 or higher
Flash Media Server 4.0 or higher with Apache 2.2 installed and configured for HTTP Dynamic Streaming
Flex 4.6 SDK or higher
OSMF 1.6 or higher
The LateBindingAudio_VOD.fxp Flash Builder project (included with this article’s sample files)
A video encoded to a .f4v, .flv, or other .mp4-compatible format (or use Obama.f4v, included with the sample files)
A separate audio track encoded to a .f4v, .flv, or other .mp4-compatible format (or use Spanish_ALT_Audio.mp4, included with the sample files)

Process overview

After completing the steps in this article you should have a good understanding of what it takes to stream on-demand video with alternate audio tracks over HTTP. At a high level, this process includes:

Packaging your media files into segments and fragments (.f4x)
Creating a master (set-level) manifest file (.f4m)
Editing the media tags for the alternate audio tracks within the master .f4m to prepare them for late-binding audio
Uploading the packaged files to Adobe Media Server
Including a cross-domain.xml file on the server if the media player is hosted on a separate domain from Adobe Media Server
Playing back the video using the master .f4m as the video source, and switching between audio tracks using OSMF

Packaging the media files

When streaming media using HDS, files first need to be packaged into segments and fragments (.f4f), index files (.f4x), and a manifest file (.f4m). Adobe Media Server 5.0 or later can automatically package your media files for both normal on-demand and live streaming with the included Live Packager application (live), and JIT HTTP Apache module (vod). However, to support late-binding audio, the manifest for the main video file must be modified to include information about the alternate audio tracks.

To package media that supports late-binding audio, you use f4fpackager, a command line tool built into Adobe Media Server. This tool accepts .f4v, .flv, or other mp4-compatible files, and is located in the rootinstall/tools/f4fpackager folder within Adobe Media Server.

In the next section, you will use f4fpackager to create packaged media files. You can use your own video and audio assets for this, or you can use the Obama.f4v (video) and Spanish_ALT_Audio.mp4 (alternate audio) files included with the sample files.

Running f4fpackager

The processes for packaging media files on Windows and Linux are similar:

From the command line target the f4fpackager executable for execution in [Adobe Media Server Install Dir]/tools/f4fpackager (Windows), or set LD_LIBRARY_PATH to the directory containing the File Packager libraries (Linux).
At the command line, type f4fpackager along with any arguments. For this example, you’ll only need to provide the following arguments for each input file:
- The name of the input file
- The file’s overall bitrate (alternatively, you could add this information manually later)
- The directory where you’d like the packaged files to be placed (if you omit this argument, the tool simply places the packaged files in the same directory as the source files)
Run the packager on the main video .f4v file (see Figure 1); for example:

        C:\Program Files\Adobe\Adobe Media Server 5\tools\f4fpackager\f4fpackager –-input-file=”C:\Obama.f4v” –-bitrate=”546” –-output-path=”E:\packaged_files”
    

Now, run the packager again, this time to package the alternate audio track; for example:

        C:\Program Files\Adobe\Adobe Media Server 5\tools\f4fpackager\f4fpackager –-input-file=”C:\Spanish_ALT_Audio.mp4” –-bitrate=”209” –-output-path=”E:\packaged_files”
    

Note: Individual media files are packaged separately, so you need to run the packager once for the main video file (Obama.f4v) and then again for the alternate audio file (Spanish_ALT_Audio.mp4).

Figure 1. Packaging media with the f4fpackager tool

The packaged output files generated by f4fpackager will be placed in the directory you specified. If you used the sample files, you should see the following output files:

Obama.f4m
ObamaSeg1.f4f
ObamaSeg1.f4x
Spanish_ALT_Audio.f4m
Spanish_ALT_AudioSeg1.f4f
Spanish_ALT_AudioSeg1.f4x

Creating the master manifest file

Next, you will need to modify Obama.f4m to create a master (set-level) manifest file that will reference the alternate audio track.

Using a text editor, open the Spanish_ALT_Audio.f4m file.
Note: If you skipped the previous section on packaging media, you can use the manifests included with the sample files in LBA/_START_/PackagedMediaFiles_START.
Select and copy everything within the bootstrapInfo and media tags.

        <?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0">
	<id>
		Spanish_ALT_Audio
	</id>
	<streamType>
		recorded
	</streamType>
	<duration>
		133.90933333333334
	</duration>
	<bootstrapInfo
		 profile="named"
		 id="bootstrap4940"
	>
		AAAB+2Fic3QAAAAAAAAAGQAAAAPoAAAAAAACCwAAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAACIBAAABtmFmcnQAAAAAAAAD6AAAAAAaAAAAAQAAAAAAAAAAAAARMAAAAAMAAAAAAAAiaAAADOQAAAAEAAAAAAAAL1AAABEwAAAABgAAAAAAAFG4AAAM5AAAAAcAAAAAAABeoAAAETAAAAAJAAAAAAAAgQkAAAzkAAAACgAAAAAAAI3wAAARMAAAAAsAAAAAAACfJAAADOQAAAAMAAAAAAAArAwAABEwAAAADgAAAAAAAM50AAAM5AAAAA8AAAAAAADbXAAAETAAAAARAAAAAAAA/cQAAAzkAAAAEgAAAAAAAQqsAAARMAAAABQAAAAAAAEtFQAADOQAAAAVAAAAAAABOfwAABEwAAAAFgAAAAAAAUswAAAM5AAAABcAAAAAAAFYGAAAETAAAAAZAAAAAAABeoAAAAzkAAAAGgAAAAAAAYdoAAARMAAAABwAAAAAAAGp0QAADOQAAAAdAAAAAAABtrgAABEwAAAAHgAAAAAAAcfsAAAM5AAAAB8AAAAAAAHU1AAAETAAAAAhAAAAAAAB9zwAAAzkAAAAIgAAAAAAAgQkAAAHCAAAAAAAAAAAAAAAAAAAAAAA
	</bootstrapInfo>
	<media
		 streamId="Spanish_ALT_Audio"
		 url="Spanish_ALT_Audio"
		 bitrate="209"
		 bootstrapInfoId="bootstrap4940"
	>
		<metadata>
			AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdGlvbgBAYL0ZQjf6igAFd2lkdGgAQIaAAAAAAAAABmhlaWdodABAfgAAAAAAAAAMdmlkZW9jb2RlY2lkAgAEYXZjMQAMYXVkaW9jb2RlY2lkAgAEbXA0YQAKYXZjcHJvZmlsZQBAU0AAAAAAAAAIYXZjbGV2ZWwAQD8AAAAAAAAADnZpZGVvZnJhbWVyYXRlAEA9+FHrhR64AA9hdWRpb3NhbXBsZXJhdGUAQOdwAAAAAAAADWF1ZGlvY2hhbm5lbHMAQAAAAAAAAAAACXRyYWNraW5mbwoAAAACAwAGbGVuZ3RoAEFOm/AAAAAAAAl0aW1lc2NhbGUAQN1EgAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAwAGbGVuZ3RoAEFYgwaAAAAAAAl0aW1lc2NhbGUAQOdwAAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAAAJ
		</metadata>
	</media>
</manifest>

    

Paste the bootstrapInfo and media tags into Obama.f4m to reference the Spanish language track.

        <?xml version="1.0" encoding="UTF-8"?>
<manifest xmlns="http://ns.adobe.com/f4m/1.0">
	<id>
		Obama
	</id>

	<streamType>
		recorded
	</streamType>

	<duration>
		133
	</duration>

	<bootstrapInfo
		 profile="named"
		 id="bootstrap4744"
	>
		AAAAq2Fic3QAAAAAAAAABAAAAAPoAAAAAAACB3QAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAACIBAAAAZmFmcnQAAAAAAAAD6AAAAAAFAAAAAQAAAAAAAAAAAAAUtAAAAAIAAAAAAAAUoAAAD6AAAAAcAAAAAAABptAAAA+gAAAAIgAAAAAAAgOgAAAD6AAAAAAAAAAAAAAAAAAAAAAA
	</bootstrapInfo>

	<media
		 streamId="Obama"
		 url="Obama"
		 bitrate="546"
		 bootstrapInfoId="bootstrap4744"
	>
		<metadata>
			AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdGlvbgBAYKAAAAAAAAAFd2lkdGgAQIaAAAAAAAAABmhlaWdodABAfgAAAAAAAAAMdmlkZW9jb2RlY2lkAgAEYXZjMQAMYXVkaW9jb2RlY2lkAgAEbXA0YQAKYXZjcHJvZmlsZQBAU0AAAAAAAAAIYXZjbGV2ZWwAQD8AAAAAAAAADnZpZGVvZnJhbWVyYXRlAEA5AAAAAAAAAA9hdWRpb3NhbXBsZXJhdGUAQOWIgAAAAAAADWF1ZGlvY2hhbm5lbHMAP/AAAAAAAAAACXRyYWNraW5mbwoAAAACAwAGbGVuZ3RoAEFm1LoAAAAAAAl0aW1lc2NhbGUAQPX5AAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAwAGbGVuZ3RoAEFWX9UAAAAAAAl0aW1lc2NhbGUAQOWIgAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAAAJ
		</metadata>

	</media>

		<bootstrapInfo
		 profile="named"
		 id="bootstrap4940"
	>
		AAAB+2Fic3QAAAAAAAAAGQAAAAPoAAAAAAACCwAAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAACIBAAABtmFmcnQAAAAAAAAD6AAAAAAaAAAAAQAAAAAAAAAAAAARMAAAAAMAAAAAAAAiaAAADOQAAAAEAAAAAAAAL1AAABEwAAAABgAAAAAAAFG4AAAM5AAAAAcAAAAAAABeoAAAETAAAAAJAAAAAAAAgQkAAAzkAAAACgAAAAAAAI3wAAARMAAAAAsAAAAAAACfJAAADOQAAAAMAAAAAAAArAwAABEwAAAADgAAAAAAAM50AAAM5AAAAA8AAAAAAADbXAAAETAAAAARAAAAAAAA/cQAAAzkAAAAEgAAAAAAAQqsAAARMAAAABQAAAAAAAEtFQAADOQAAAAVAAAAAAABOfwAABEwAAAAFgAAAAAAAUswAAAM5AAAABcAAAAAAAFYGAAAETAAAAAZAAAAAAABeoAAAAzkAAAAGgAAAAAAAYdoAAARMAAAABwAAAAAAAGp0QAADOQAAAAdAAAAAAABtrgAABEwAAAAHgAAAAAAAcfsAAAM5AAAAB8AAAAAAAHU1AAAETAAAAAhAAAAAAAB9zwAAAzkAAAAIgAAAAAAAgQkAAAHCAAAAAAAAAAAAAAAAAAAAAAA
	</bootstrapInfo>

	<media
		 streamId="Spanish_ALT_Audio"
		 url="Spanish_ALT_Audio"
		 bitrate="209"
		 bootstrapInfoId="bootstrap4940"
	>
		<metadata>
			AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdGlvbgBAYL0ZQjf6igAFd2lkdGgAQIaAAAAAAAAABmhlaWdodABAfgAAAAAAAAAMdmlkZW9jb2RlY2lkAgAEYXZjMQAMYXVkaW9jb2RlY2lkAgAEbXA0YQAKYXZjcHJvZmlsZQBAU0AAAAAAAAAIYXZjbGV2ZWwAQD8AAAAAAAAADnZpZGVvZnJhbWVyYXRlAEA9+FHrhR64AA9hdWRpb3NhbXBsZXJhdGUAQOdwAAAAAAAADWF1ZGlvY2hhbm5lbHMAQAAAAAAAAAAACXRyYWNraW5mbwoAAAACAwAGbGVuZ3RoAEFOm/AAAAAAAAl0aW1lc2NhbGUAQN1EgAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAwAGbGVuZ3RoAEFYgwaAAAAAAAl0aW1lc2NhbGUAQOdwAAAAAAAACGxhbmd1YWdlAgADZW5nAAAJAAAJ
		</metadata>

	</media>
	
</manifest>
    

Add the following attributes to the media tag for the Spanish language track within Obama.f4m:
- alternate="true"
- type="audio"
- lang="Spanish"

        <media
   streamId="Spanish_ALT_Audio"
   url="Spanish_ALT_Audio"
   bitrate="209"
   bootstrapInfoId="bootstrap4940"
   alternate="true"
   type="audio"
   lang="Spanish"
>
    

The alternate="true" and type="audio" settings enable OSMF to parse Obama.f4m and see that there is alternate audio available. Logic within the included example player, which you’ll be using to play the video in a later step, uses lang="Spanish" to populate a dropdown menu with the available alternate audio stream.

Save Obama.f4m. This is now the master manifest file, and you will need to reference it to play the video and alternate audio content with OSMF.

Uploading the packaged files to the server

The next step is to upload the packaged files to a folder within the webroot/vod directory of Adobe Media Server (see Figure 2). On Windows this default location is C:\Program Files\Adobe\Adobe Media Server 5\webroot\vod. Later, you will point OSMF to the master manifest within this directory to play the video.

Figure 2. The packaged media files.

Verifying the delivery of the .f4m file

After uploading the packaged files to a directory on the server within /webroot/vod, it’s a good idea to verify that the server is delivering the manifest file properly. You can do that by typing the URL of the .f4m file in the address bar of a browser.

To test the delivery of the manifest, open the .f4m in a browser via your web server. On a local development machine the URL would be something like:

http://127.0.0.1/vod/Obama.f4m

If you’ve entered the URL correctly, and the server is properly serving the .f4m file, you should see the manifest’s xml.

Note: Safari will not display XML by default.

Notice the alternate audio’s media and bootstrap info tags you added earlier, as well as the additional attributes in the media tag:

        <manifest xmlns="http://ns.adobe.com/f4m/1.0">

<id>Obama</id>
<streamType>recorded</streamType>
<duration>133</duration>

<bootstrapInfo profile="named" id="bootstrap4744">
AAAAq2Fic3QAAAAAAAAABAAAAAPoAAAAAAACB3QAAA…
</bootstrapInfo>

<media streamId="Obama" url="Obama" bitrate="546" 
bootstrapInfoId="bootstrap4744">
<metadata>
AgAKb25NZXRhRGF0YQgAAAAAAAhkd…
</metadata>
</media>

<bootstrapInfo profile="named" id="bootstrap4940">
AAAB+2Fic3QAAAAAAAAAGQAAAAPoAAAAAAACCwAAAAAA…
</bootstrapInfo>

<media streamId="Spanish_ALT_Audio" url="Spanish_ALT_Audio" bitrate="209" bootstrapInfoId="bootstrap4940" alternate="true" type="audio" lang="Spanish">
<metadata>
AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJh…
</metadata>
</media>

</manifest>
    

Note: The example URL used above does not point to /hds-vod like it would for HDS content that needs to be packaged just-in-time as the client application requests it. This is because /hds-vod is a location directive for Apache that tells the server to look for content in the /webroot/vod directory and package it for delivery. The jithttp module in Apache responsible for just-in-time packaging isn’t needed for this example, as the source files have been already packaged manually.

Including a cross-domain policy file (optional)

To access content from Adobe Media Server using a Flash-based media player that is hosted on a separate domain from the server, the player needs permission in the form of a cross-domain policy file hosted on the server. Figure 3 shows an example of a cross-domain policy file that allows access from any domain. You may want to use a more restrictive cross-domain policy for security reasons. For more information on cross-domain policy files, see Setting a crossdomain.xml file for HTTP streaming.

Figure 3. A crossdomain.xml file in the webroot directory of Adobe Media Server

Using a text editor, open crossdomain.xml in the LBA/_START_/ folder of the sample files.
Examine the permissions, and edit them if you wish to limit access to include only specific domains.
Save the file, and upload it to the /webroot directory of Adobe Media Server.

Testing the video using an OSMF-based media player

Now you are ready to test the video and the alternate audio streams using the included sample media player. The sample player is provided as a Flash Builder project, and is based on the LateBindingAudioSample application that comes as part of the OSMF source download (OSMF/samples/LateBindingAudioSample).

You can find the included sample player in the LBA/_COMPLETED_/LateBindingAudio_VOD.fxp Flex project file among the sample files.

Import LateBindingAudio_VOD.fxp into Flash Builder and run the project.
For the Media Source setting, type the URL of the master manifest located on your server.
If the player can access the .f4m file, the Play button will be enabled, and the alternate languages dropdown menu will show a Spanish audio option.
Click Play and then choose Spanish as the alternate language (see Figure 4).

The video should start to play, and you should see Switching Audio Track displayed in the player under the languages menu. The audio should then switch to the Spanish track, while the video continues to play normally.

Figure 4. The media playing within the included sample application.

Where to go from here

This article covered the steps necessary to prepare and deliver files to attach late-binding audio to an HTTP video stream using a prebuilt OSMF-based media player. You should now have a good understanding of how to package media files for delivery over HTTP, as well as what needs to be done on the server side to deliver late-binding audio.

In addition to on-demand video, OSMF supports late-binding audio for live video, multi-bitrate video, and DVR. For more information about HTTP Dynamic Streaming, late-binding audio, and OSMF, refer to the following resources:

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

Tutorials & Samples

Tutorials

Samples

Flash Media Server Forum

05/22/2013	Streaming capabilities of Adobe Media Server, thank you for your help
05/22/2013	Re-streaming /streams/_definst_/*.f4f fragments with hls-vod or hds-vod buffer every 8 seconds! Why?
05/22/2013	AMS 5 VOD streaming to Android
05/22/2013	Using UNC path from Client in VOD application

Attaching alternate audio tracks to HTTP video streams using OSMF

by Scott Sheridan

Content

Created

Page tools

Requirements

Prerequisite knowledge

Additional required other products

User level

Required products

Sample files