Pixel Bender language overview

Before jumping into kernel coding, let's take a brief look at the Pixel Bender language. You probably won't become an expert in the language just from reading this article, but the following discussion should help you understand programs written in the Pixel Bender language. For far more detail, refer to the Adobe Pixel Bender Language 1.0 Tutorial and Reference Guide, available from the Help menu of the Pixel Bender Toolkit application.

Pixel Bender uses procedural syntax similar to languages such as C, Java, and ActionScript. It includes built-in data types and functions targeted at image processing. If you are already familiar with ActionScript, then learning to write Pixel Bender kernels should be reasonably straightforward. The main differences in syntax between ActionScript and Pixel Bender language include:

• Type declarations for variables go in front of the variable name, instead of after it. The var keyword is not used. Thus, instead of declaring an integer value with:

  var foo:int;
            

  you use the syntax:

  int foo;

• New objects cannot be created at runtime. Thus, the new keyword is not supported (or needed).
• Pixel Bender uses the data type, float, instead of the data type, Number, for representing real numbers. The term float stands for floating-point (which just means that the decimal point can "float" to any position in the number).
• It is important to remember that when you type a floating point number in a Pixel Bender program, you must include a decimal point. Otherwise, the Pixel Bender compiler will treat the number as an integer rather than a float value. Unlike ActionScript, Pixel Bender does not do any implicit data conversion. So if, for example, you type 1 instead of 1.0, you will get an abstruse error message from the Pixel Bender Toolkit.
• Pixel Bender includes built-in vector data types. A vector type can be recognized by the number appended to the base type name. For example, a float3 type is a vector containing three elements of type float.

• The following statement illustrates a typical use of a vector data type in Pixel Bender:

  float4 rgbaPixel = float4( 1.0, 0.3, 0.2, 0.8 );

• The statement above declares a new vector variable named rgbaPixel and assigns it a color value. The expression, float4( 1.0, 0.3, 0.2, 0.8 ), defines a literal float4 vector constant, which is assigned to the rgbaPixel variable.
• Pixel Bender supports vector swizzling.

  The members of a vector can be accessed using dot notation and one of three sets of elements names: r,g,b,a; x,y,z,w; or s,t,p,q. Swizzling lets you rearrange the elements simply by reordering the element names. For example, the following statement swaps the red and green color channels of a pixel vector when assigning the pixel value to another variable:

  pixel4 mixedUp = rgbaPixel.grba;

  You can also repeat channels:

  pixel4 allRed = rgbaPixel.rrra;

  and drop channels:

  pixel2 redAndBlue = rgbaPixel.rb;

  Note: The choice of which set of element names to use with a vector variable is up to you. For example, myVector.r is the same as myVector.x. A good practice is to use the rgba set for colors, and the xyzw or stpq sets for positions. You cannot mix names from more than one set in the same reference.

• Pixel Bender also supports built-in matrix data types. A matrix data type is similar to a vector, but contains a two dimensional array of numbers. The float3x3 matrix type, for example, contains three vectors of three elements.
  
  
• Pixel Bender in Flash Player does not support loops, custom functions (other than the evaluatePixel() function), or arrays.

Note: When developing Pixel Bender kernels for Flash Player, be sure to enable the Turn on Flash Player Warnings and Errors option (under the Build menu of the toolkit window). With this option enabled, the compiler will inform you immediately when you are using unsupported Pixel Bender language features. (Otherwise, you won't see the errors until you try to export the kernel for Flash Player.)

Kernel walkthrough

The typical Pixel bender kernel performs the following tasks:

• Samples a pixel from the input image
• Applies a calculation to the sampled pixel color
• Assigns the modified value to the output pixel

The following simple Pixel Bender kernel does each of these tasks. The program defines a kernel named ChannelScrambler:

<languageVersion : 1.0;>

kernel ChannelScramblerFilter
<   namespace : "com.adobe.example";
    vendor : "Adobe Systems Inc.";
    version : 1;
    description : "Changes the color channel order from rgba to gbra.";
>
{
    input image4 inputImage;
    output pixel4 outPixel;

    void evaluatePixel()
    {
        pixel4 samplePixel = sampleNearest( inputImage, outCoord() );
        outPixel = samplePixel.gbra;
    }
}
          

The kernel declares an input image, named inputImage, and an output pixel, named outPixel. In the evaluatePixel() function, the pixel at the output coordinates currently being processed is accessed using two built-in functions, sampleNearest() and outCoord(). Next, the sampled pixel is assigned to the outPixel variable, using swizzling to reorder the color channels.

This kernel produces the following result when used as a filter in SWF files generated in Flash CS4:

The required elements of a Pixel Bender program include the languageVersion tag:

<languageVersion: 1.0;> 

and the kernel name declaration:

kernel ChannelScrambler{...} 

Inside the kernel, there must be a single output declaration, such as output pixel4 outPixel, and an evaluatePixel() function. A kernel may have any number of inputs, including no inputs at all. However, the way you use a kernel in Flash Player creates additional requirements on the number of inputs. A shader used as a blend requires two inputs, a shader used as a filter requires one input, and a shader used as a fill does not require any inputs at all.

A Pixel Bender kernel is run once for every pixel in the output image. No state information is saved between runs, so it isn't possible to collect an average pixel value by accumulating the pixels sampled in each run. Each run of the kernel must compute all the information it needs—although you can pre-compute information in ActionScript and pass it in as an input or parameter.

Sampling

To access the pixel values in an input image, you must use the sampling functions. Sampling is accomplished with the following built-in functions:

• sampleNearest() returns a vector containing the channel values of the pixel closest to the specified coordinates. (There is also a sampleLinear() function that behaves slightly differently.)

• outCoord() returns the coordinates of the current output pixel.

As Pixel Bender processes an image, it executes the kernel for every pixel in the output image. The outCoord() function returns the coordinates of the current pixel. The Pixel Bender coordinate system is similar to that used by Flash Player. The origin is registered at the top, left corner. Positive values increase to the right and down. Pixels are always square.

You are not limited to sampling the pixel directly at the outCoord() position. For example, you could use the following sampling statement to sample the pixel that is 10 pixels right and 5 pixels down from the current pixel:

pixel4 inputPixel = sampleNearest( inputImage, outCoord() + float2( 10.0, 5.0 ) );

The expression, outCoord() + float2( 10.0, 5.0 ), adds a two-element vector to the vector of coordinates produced by the outCoord() function. An equivalent way to code the same expression is: float2(outCoord().x + 10.0, outCoord().y + 5.0).

If the sample coordinates go beyond the bounds of the input image, then a color vector containing all zeros is returned. For example, if the input is of type image4, then sampling an exterior pixel will return a completely transparent black pixel. If the input is of type image3, then it will return a black pixel (without an alpha channel). The Pixel Bender coordinate space theoretically extends to infinity. There are, of course, practical limits on the range of coordinates that can be expressed, as well as limits on the usefulness of sampling data that does not exist.

Working with pixels

Once you have a vector representing a pixel, there are a number of ways to work with the color values. For example, if you have a pixel4 variable named pix, you can address the individual color channels, or combinations of channels in the following ways:

• Red channel: pix.r or pix[0]
• Green channel: pix.g or pix[1]
• Blue channel: pix.b or pix[2]
• Alpha channel: pix.a or pix[3]
• Red and alpha channels: pix.ra
• All the color channels with red and blue swapped and no alpha channel: pix.bgr

A single color channel is a 32-bit floating-point number, normally between 0.0 (black) and 1.0 (white). The output color values can be outside this range, but they won't change the rendered appearance. In other words, pixel3(-1.0, -1.0, -1.0) is just as black as pixel3(0.0, 0.0, 0.0) when rendered as a bitmap. (However, if you run multiple filters on the same image, the difference can be significant since the second filter will see (-1.0, -1.0, -1.0) not (0.0, 0.0, 0.0) when sampling that pixel.)

You can perform arithmetic on a pixel vector with either scalar or vector values. When using scalar values, the operation is applied to each channel. For example, the following operation will divide the value of each channel in half (including the alpha channel):

pixel4 pix = sampleNearest( inputImage, outCoord()); 
pix = pix / 2.0;
          

The same operation could also be written using vectors:

pix = pix * pixel4( 0.5, 0.5, 0.5, 0.5 );

Although Pixel Bender images have 32-bits per channel, graphics in Flash CS4 only have 8-bits per channel. When a kernel is run in Flash Player, the input image data is converted to 32-bits per channel and then converted back to 8-bits per channel when kernel execution is complete.

Defining inputs

Inputs are declared with the input keyword:

input image4 sourceImage;

Inputs can be declared using the data types: image1, image2, image3, or image4. The different inputs in a kernel can have different numbers of channels. The number of channels in the output image produced by the shader is determined by the data type of the output pixel, not by the data types of the inputs.

You can declare more inputs than are required for the way a kernel is used in Flash Player. However, Flash Player will only automatically assign image data to the required inputs. You must assign an image to the extra inputs before assigning the kernel as a blend, filter or fill. For example, if your filter kernel used an additional image to create a textured effect, you would have to assign the texture as an input to the kernel before assigning the ShaderFilter containing the kernel to the filter array of a display object (more on how to do that later in this article).

Defining parameters

In addition to input images, you can supply other values to a kernel as parameters. Parameters are declared with the parameter keyword and can be any data type, except image (or region—which you can't use in kernels written for Flash Player anyway). You can declare metadata for parameters to specify the default, minimum, and maximum values. You can also supply a description. The metadata is declared between angle braces (<>) and can be accessed in Flash Player. It is always a good idea to define a reasonable default value for a parameter.

The following parameter statement declares a float3 parameter with metadata:

parameter float3 weights
<
     defaultValue : float3( 0.5, 0.5, 0.5 );     
     minValue :   float3( 0.1, 0.1, 0.1 );     
     maxValue :  float3( 0.9, 0.9, 0.9 );     
     description : "A three element vector of weight values." 
>;
          

To access parameter values in Flash Player, you use the data property of the ActionScript Shader object containing the kernel. The current, minimum and maximum values of the above parameter can be accessed in ActionScript with the following statements (assuming that myShader is a Shader object containing a kernel with this parameter):

var currentWeights:Array = myShader.data.weights.value; 
var minWeights:Array = myShader.data.weights.minimumValue; 
var maxWeights:Array = myShader.data.weights.maximumValue;
          

Since the parameter is a float3 vector type, the ActionScript arrays returned will contain three elements. If the parameter was a scalar type, such as float, then the arrays returned would contain a single element.

Page 2 of 6

Previous Page

Next Page