Requirements

Prerequisite knowledge

Basic familiarity with the Stage3D API is required. Prior knowledge of working with Stage3D is recommended.

User level

Beginning

 

Required products

Flash Builder (Download trial)

Adobe Animate CC

 

Sample files

Sample files

 
Video Texture Overview

Before AIR 17, the use of video in Stage3D required the use of the Video object (which is not hardware accelerated), copying of video frame to a BitmapData object, and loading of the data onto the GPU which is CPU intensive .
 
So,Video texture object was introduced, which allows hardware decoded video to be used in Stage 3D content. Video Texture is only available for AIR not for Flash Player .
 

 
How video was rendered using Video Object

ActionScript (AS) does not allow direct access to texture from video or camera for use with Stage3D. 
 
In order to use video or camera frames as textures with stage3D, the SWF content developer must use a bitmap representation of a video or camera frame.
 
Once a bitmap is created from the video or camera object, the developer needs to create a texture and update the created texture with the bitmap. 
 
This process normally requires multiple copying of the bitmap, which can be slow and resource intensive.
 
Pictorial representation of the process

Pictorial representation of the process

 

 
New Approach: Video Texture Object

To resolve this problem, video texture objects were added to support NetStream and Cameras in a manner consistent/similar to StageVideo. 
 
Such textures can be used as source textures in stage3D rendering pipeline. The textures can be used as rectangular, RGB, no mipmap textures in the rendering of a scene. They are treated as ARGB texture by the shaders (that is, the AGAL shaders do not have to bother about YUV->RGB conversion) so standard shaders used with static images can be used without change. The image that is used by the rendering pipeline is the most up-to-date frame at the time the rendering using this texture occurs. There is no tearing in a video frame, but if the same texture is used several times, some of the instances may be from different timestamps.
 
Using a VideoTexture object, all this work is optimized internally and  YUV to RGB conversion and texture loading can be completely moved to the GPU.
 

 
Use cases

  • Video transitions in game. Often games use video to transition between story lines in the game.
    Currently the best way to do this is to use StageVideo. However if StageVideo is used for this purpose, the stage3D content needs to be removed so that it does not obsure the StageVideo.
     
    Hence Stage3D content cannot be used as an overlay. Video Texture allows efficient use of Video textures with Stage3D.
     
  • Remote/Local camera in game (such as chatting with game mates), or applying shader effects to a video camera stream.
    Video Texture allows you to efficiently embed camera streams within Stage3D content.
     
  • Game video streams: Use of video streams inside the content (for example inside a monitor in the scene).
  • Applying special effects to video (3D vision for example, like YouTube does with Hydra Software Shaders) and still get full HW acceleration, even if it not as efficient as StageVideo, it will still be GPU accelerated.

 
Using Video Texture in ActionScript

There are three steps to use Video Texture in action script.  
 
Step 1:
 
Confirm if VideoTexture is supported by Context3D class
Before using VideoTexture object, a check should be made to confirm that Context3D class supports VideoTexture by checking  a static property "supportsVideoTexture" of Context3D class.
 
if( stage.stage3Ds.length > 0 ) { var stage3D:Stage3D = stage.stage3Ds[0]; stage3D.addEventListener( Event.CONTEXT3D_CREATE, myContext3DHandler ); stage3D.requestContext3D( ); } function myContext3DHandler ( event : Event ) : void { if (Context3D.supportsVideoTexture) { // We can create VideoTexture Object with Context3D object. } }
Step 2:
 
Create VideoTexture object and attach NetStream object to created VideoTexture object.
Create a VideoTexture object and attach a NetStream/Camera object to the object. Ensure that event listener is added before calling NetStream.play for Event.TEXTURE_READY to prevent the event from being invoked when there is no registered event listener.
 
//Connections var nc:NetConnection; var ns:NetStream; nc=new NetConnection(); nc.connect(null); ns=new NetStream(nc); var texture:VideoTexture; texture = context3D.createVideoTexture(); texture.attachNetstream(ns); texture.addEventListener(Event.TEXTURE_READY, renderFrame); // Need to add event listener before calling NetStream.play ns.play("movie.mp4");
Step 3:
 
Use an updated VideoTexture object with currently available video frame in attached NetStream object.
Retrieve the current video frame from the VideoTexture object when the callback for the Event.TEXTURE_READY event is fired.
 
Event.TEXTURE_READY is fired when a video frame corresponding to current audio playback is available.  Video frames not in sync with the current audio playback may be dropped.
 
var context3D:Context3D; function renderFrame(e:Event):void { // Render on stage3D with VideoTexture }
Through the VideoTextureEvent object, the VideoTexture feature provides notification of the same events as that for the StageVideoEvent object.
 
texture.addEventListener(VideoTextureEvent.RENDER_STATE, onRenderState); function onRenderState (event:VideoTextureEvent) { if ( event.status == VideoStatus.SOFTWARE) { // Indicates software video decoding works. B } if ( event.status == VideoStatus.ACCELERATED ) { // Indicates hardware-accelerated (GPU) video decoding works. } if ( event.status == VideoStatus.UNAVAILABLE ) { // Indicates Video decoding is not supported. } }

 
Video Texture limitations

Video Texture has the following limitations:
 
  • VideoTexture feature does not support software rendering mode of Context3D.
    The Context3D.CreateVideoTexture method throws an error “Texture Creation Failed. Internal error.” if the method is called in software rendering mode.
  • VideoTexture object has a limitation on texture sampling. Here are the available options for VideoTexture object:
    • Texture dimension. Options: 2d
    • Mip mapping. Options: nomip (or mipnone)
    • Texture repeat. Options: clamp
  • Total available VideoTexture object in context3d is 4.
 

 
Sample project

You can download the sample project (VideoTextureSample.zip attached on top of this page) and try running the app on desktop/mobile.
 

 
Where to go from here

Refer to Video Texture in ActionScript 3.0 Reference for the Adobe Flash Platform for more information on Video Texture object, its properties, methods, and events.