Adding real-time data visualization to your ColdFusion applications using Flex

In this article, I will share what I’ve learned in the past few months about using ColdFusion as a source of real-time data and using Flex combined with Google Maps API for Flash as a means of visualizing that data. The live map below is an example of what you can build using this technique. (If you can’t see the live map, make sure you have Flash enabled). The map provides a real-time view of activity on Flex.org, my blog, and the blogs of several colleagues including James Ward, Christophe Coenraets, Michael Chaize, Anne Petteroe, Holly Schinsky, Ted Patrick, Kevin Hoyt, Ryan Stewart, and Ray Camden. Every time one of these sites is viewed by someone, a dot appears on the map at their approximate location. 

Try it out for yourself. Click Load Last Hour to replay the last hour in accelerated time. I’ll let you figure out which color marker goes with each blog.

The easiest way to tap into real-time data is to employ a messaging paradigm called publish/subscribe messaging (commonly referred to as pub/sub).  In the application above, the website and blog activity is published to a message destination as small messages.  This article covers everything you need to add this kind of real-time data visualization to your website or application using pub/sub messaging.  It explains the ColdFusion code that publishes data as well as the Flex code that subscribes to these published messages and displays the data in interesting ways.

I used the following technologies to build this application:

• ColdFusion 9.01 standalone running on port 8500—This is a fresh out-of-the-box install on Windows 7. (The application will also work with ColdFusion 8.) The instructions in this article are written based on a ColdFusion installation  in C:\ColdFusion9.  If you’re not using Windows or if you installed ColdFusion in a different directory, adjust the paths accordingly.
• BlazeDS 4—This is included with ColdFusion 9.01. The application also works with BlazeDS 3 and LiveCycle Data Services.
• Flex SDK 4.1—All of this will work with Flex 3.x or Flex 4.0 too with minor changes.
• Flash Builder 4
• ColdFusion Builder

ColdFusion uses the sendGatewayMessage() function to publish messages. Before you can use this nifty function, you need to first configure a data service messaging gateway instance:

1. Start ColdFusion administrator.
2. Click Event Gateways and then click Gateway Instances.
3. On the Gateway Instances page, specify a name for the new gateway; for example type cfgw as the Gateway ID.
4. Select DataServicesMessaging as the Gateway Type.
5. For now, just point the CFC PATH to any CFC.

Note: By creating an onMessage() function, ColdFusion can receive messages through the gateway. For this article, however, you're only going to focus on sending messages so it's not necessary to have this function.

6. For the Configuration File, select C:\ColdFusion9\gateway\config\flex-messaging-gateway.cfg (see Figure 1).
7. Click Add Gateway Instance create the instance.
8. Edit C:\ColdFusion9\gateway\config\flex-messaging-gateway.cfg) and comment out the following line:

See the comments in the flex-messaging-gateway.cfg file for more details on this.

9. Restart ColdFusion. Check the cfserver.log for any gateway-related errors.

Now you can use the sendGatewayMessage() function to broadcast data. The code below publishes a simple struct that contains "Hello World" in the body.

Notice the sendGatewayMessage() function uses the recently created cfgw gateway instance. Also, notice the destination . This example uses the preconfigured destination named ColdFusionGateway . If you want to see the definition of this destination, check out the following two files:

• c:\ColdFusion9\wwwroot\WEB-INF\flex\messaging-config.xml — This file defines messaging destinations; you'll see ColdFusionGateway already defined. Near the bottom of the definition is a section for configuring the communication channel:

The cf-polling-amf channel is defined in the services-config.xml file.

• c:\ColdFusion9\wwwroot\WEB-INF\flex\services-config.xml — This file defines the individual services.

I'll provide more details on these configuration files in Getting technical: Channel types.

Now you have some simple ColdFusion code that publishes messages.  Next, you need to build something to consume them.

Below is code for a basic Flex application that will subscribe to the same destination defined above, ColdFusionGateway. To try this out, simply create a new Flex project in Flash Builder and copy the code below into the main MXML file.  Don’t worry about selecting an application server in this sample because you’re defining everything in the code.  Sometimes it’s just easier for me to understand everything when it is there in front of me rather than wired into a properties file somewhere.

The only thing you’ll need to change is the IP address in the ChannelSet to point to your ColdFusion server.  The code includes event handlers that fire when the application successfully connects to the server, when a message is received, and when the application gets disconnected from the server.  When you build and run the application you should see the "Channel Connected!" message after a few seconds. 

Now you can run the ColdFusion code above to send the "Hello world" message and you should see it received in your Flex application.  To run the code, simply paste it into a CFM file in your wwwroot directory and then access that file from a browser. It might take a few seconds to see the message because of the type of channel you are using.  In Getting technical: Channel types I’ll explain how to improve the response time by creating our own custom channel definition.

Now that you know how to publish data from ColdFusion and consume that data from a Flex application, it's easy to build some really cool stuff. To begin, you'll need to add some additional data to your outbound message from ColdFusion.

Getting the IP address in ColdFusion is straightforward. It's already available to you in #CGI.REMOTE_ADDR# . There are several services that provide IP address to location conversion but my favorite way to do it is to use a local database from MaxMind, a provider of geolocation and online fraud detection tools. The conversion is lightning fast because the database is right on your server. They have several databases available, but I highly recommend using one of the following:

• MaxMind GeoLite City—This is a free version. It's not as accurate as the paid version but it's great for testing or for basic traffic analysis.
• MaxMind GeoIP City—This is a more accurate commercial version of the database that is a drop-in replacement for GeoLite City.

MaxMind provides APIs for C, C#, Perl, PHP, Java, Python, Ruby, VB.NET, JavaScript and even Pascal. Since Java is in the list, it's easy to get this working with ColdFusion as well. Thankfully, I didn't have to do any work on this because Brandon Purcell has already done it and provided step-by-step instructions on his blog. To summarize Brandon's steps:

1. Download a JAR file that he created (geoIP.jar).
2. Copy it to your C:\ColdFusion9\runtime\servers\lib folder.
3. Restart ColdFusion.
4. Add some code to your Application.cfc (an example is provided in Brandon's post).
5. Put the included geo.cfc file in your application folder.

Now you can use the database to send geographical data via the gateway:

Notice that I added a <cfdump> to the end so I can see the results. It's very likely that if you run the code above, it will not show a location. This is because if you are in the same subnet as your server your IP address is private. To test the code, you can use your public IP address in place of #CGI.REMOTE_ADDR#. You should see the geodata information (see Figure 2) when you run the code.

Now the message you are broadcasting contains city, country (in two formats), latitude, and longitude. You can extract these new fields on the Flex side by referencing message.body.city , message.body.country , message.body.countryshort , message.body.latitude , and message.body.longitude .

Now that you have latitude and longitude, you can use any of several different mapping services. You'll find examples of Google, ESRI, MapQuest and Yahoo! Maps in Tour de Flex under the mapping category. In this article, you'll use the Google 3D Maps API for Flash. For information on other Google Maps APIs, visit Google Maps API Family.

I decided to use the 3D Map API for this because it allows you to change the map attitude, pitch, and yaw, which is a lot of fun and always impresses.

Here are the steps to get this working:

1. Visit the Google Maps for Flash page and sign up for a Google Maps API Key. (It takes less than a minute and it's free.)
2. Create a new Flex project as above (no special options and no application server).
3. Download the Google Maps API for Flash SDK from the same page. The ZIP file contains documentation and the map_flex_1_19a.swc file that you need to place in your Flex project's libs folder. (As of this writing the version was 1.18b; your SWC filename may vary.)
4. Copy the code below into your main MXML file:

5. Modify the IP address in the <ChannelSet/> to point to your ColdFusion server.
6. Replace "PUT-YOUR-GOOGLE-API-KEY-HERE" with the API key you obtained.
7. Build and run the new Flex application.

Every time your ColdFusion code broadcasts a new activity message, your Flex application will extract the location from the message and display it on the Google 3D map.

You'll find tons of fun features including the ability to fly to a new location, customize markers, and more in the Google Maps API ActionScript Reference. For more demonstrations of the Google Maps API for Flash see Google's Demo Gallery.

When I first started playing with real-time visualization of geolocation data, I was eager to add tracking to my own blog so I could watch my blog activity on a world map. My blog, however, is not ColdFusion based so I didn't have the luxury of adding a few lines of ColdFusion code to broadcast the needed data. There are several server-side options that could work but my blog is hosted at WordPress.com so my options were very limited. I don't even have access to the server. Late one night I had an idea—I would use the <IMG> tag! It's the most common way external assets are included in HTML.

With a little help from Ray Camden, I figured out how to get a CFM file to behave like an image. On my blog, I simply have an image as follows:

Web browsers think it's a 1×1 transparent PNG.

Here's the source for trackimg.cfm:

Using this technique, you can add tracking to any of your websites by simply adding the image tag. In the demo application, you'll notice that different sites show up as different colors. This is done by adding a color variable to the geodata struct . You set the color based on the domain in the referrer ( #CGI.HTTP_REFERRER# ) using code like this:

On the Flex side, you can use message.body.color to set the marker color. You can use message.body.domain and message.body.url in the info window. There are lots of possibilities.

All of the Flex code in this article is set up to receive all messages sent from the ColdFusion server. If you later want to use the same ColdFusion gateway for multiple applications, you're going to need some way to filter messages. This is where subtopics come in. Subtopics provide a means of categorizing messages within a destination. By default, subtopics are enabled as you can see near the top of C:\ColdFusion9\wwwroot\WEB-INF\flex\messaging-config.xml. To publish messages on a specific subtopic, add the following to your struct :

On the Flex side, you need to add subtopic= " BLOGS " to your <mx:Consumer> . Now your Flex application will only receive messages sent specifically on the BLOGS subtopic. Subtopics can be defined using the following format: mainToken[.secondaryToken][.additionalToken]. For example, you could use something like BLOGS.GREG for one site and BLOGS.BEN on another. On the Flex side, you can subscribe to either or you can use wildcards to subscribe to multiple subtopics; for example, subtopic="BLOGS.*" . When you subscribe to a subtopic, the filtering happens at the server, so your application only receives messages that match.

In addition to subtopics, you can also use selectors. For more information on selectors and subtopics, refer to the BlazeDS and LiveCycle Data Services documentation.

You may have noticed that there is sometimes a delay of a few seconds after ColdFusion sends the message until the Flex application receives it. So far you have been using the out-of-the box configuration for ColdFusionGateway . If you look in C:\ColdFusion9\wwwroot\WEB-INF\messaging-config.xml, you'll see that the assigned channel is cf-polling-amf , which is defined in C:\ColdFusion9\wwwroot\WEB-INF\flex\services-config.xml. This is set up to do basic polling, which is not very responsive. You can speed things up significantly by implementing long polling. Here are the steps:

1. Add the following channel definition to C:\Coldfusion9\wwwroot\WEB-INF\flex\services-config.xml:

2. Edit C:\ColdFusion9\wwwroot\WEB-INF\flex\messaging-config.xml and assign this new channel to the ColdFusionGateway destination. The modified XML will look like this:

3. Modify your Flex code and change the URL of your ChannelSet :

4. Restart ColdFusion and everything should work as before, but you'll notice that messages get through faster.

Polling and long polling are not the only two options. BlazeDS also supports HTTP streaming, which has even better performance. Some firewalls, however, don't like a persistent connection, which can be a problem with streaming. You can find more information about endpoints in Holly Schinsky's blog post on the topic and in the LiveCycle Data Services documentation.

LiveCycle Data Services ES2 offers the same messaging features as BlazeDS but it also offers additional capabilities that you need to be aware of.

• RTMP: In addition to polling, long polling, and HTTP streaming, LiveCycle Data Services provides Real-Time Messaging Protocol (RTMP). RTMP streams data in real-time over the TCP-based RTMP protocol in the binary AMF format. It's super fast and efficient and great for situations in which there are thousands of subscribers to your message topics.
• NIO-based endpoints: Unlike BlazeDS endpoints, NIO-based endpoints are outside the servlet container and run inside an NIO-based socket server. NIO-based endpoints can offer significant scalability gains. Because they are NIO-based, they are not limited to one thread per connection. Far fewer threads can efficiently handle high numbers of connections and IO operations. For more information on NIO-based endpoints, see About Channels and Endpoints.
• Reliable messaging: In LiveCycle Data Services, you can set a destination to be reliable. Basically, LiveCycle Data Services will confirm that each message is received by all active consumers. With non-reliable messaging, a short network outage could result in lost messages because there is no retry mechanism. For applications like the activity tracking map, this is no big deal–you'll just miss a dot on the map. However, if you use messaging in an application where every message is critical, this is a key feature. For more details, see Reliable Messaging.
• Message throttling: Throttling allows you to limit the number of server-to-client and client-to-server messages that a Message Service destination can process per second. Throttling is useful when the server could overwhelm slow clients with a high number of messages that they cannot process or when the server could be overwhelmed with messages from clients. You can read more about throttling in Data Throttling.

You can find the complete LiveCycle Data Services 3.1 documentation here.

This article has only addressed the messaging features of LiveCycle Data Services. There are many other compelling facets to LiveCycle Data Services, and I encourage you to explore them. For messaging, it's fairly simple. If you need the features I just listed or if you need highly scalable messaging, then you need LiveCycle Data Services, otherwise BlazeDS will work for you.

So, what's next for the real-time map application?

I've recently added server-side logging and the ability to replay history on the map (for example, load last hour, load last day, and so on). I've also added icons and other features.

For a real-world example of real-time data on a map, visit this page. This application shows real-time activity from ChessJam, an online chess site that I helped develop. In this case, the server is ColdFusion, so I simply added code similar to the sendGatewayMessage() code shown above in the function that handles chess moves. Every time a player makes a move, it shows on the map. I used an image of a map from 1812 as an overlay to give it a little age. The really amazing thing here is that they made a map of the world in 1812 with no GPS and no satellite pictures!