By Frank DeRienzo
Principle Support Engineer
Allaire Corp.
Does this story sound familiar? Until now, you have avoided
clustering your ColdFusion Web site, but you are experiencing
sharp peaks in load that degrade the your site's performance.
You need Web performance to create revenue, but you need
Web-generated revenue to justify the setup costs for the
increased Web presence that would create the needed revenue.
In short, you're stuck.
With Allaire ClusterCATS (CC), you can create a cluster
of low-end Web servers with nominal investment, increase
load capacity, realize improved performance, and virtually
eliminate downtime caused by server failure. ClusterCATS
redirects incoming traffic to the server with the least
amount of load. Also, if one of your Web servers fails,
the remaining server will pick up the failed server's IP
address by hosting the sessions bound for both IP addresses.
Alternatively, if one server becomes overloaded, sessions
will be redirected to the least loaded server in the cluster.
ClusterCATS also provides other features. For example,
if a Web server or an application server hangs, ClusterCATS
will restart or restrict the problematic server redirecting
traffic to the available clustered server. ClusterCATS can
also send e-mail alerts to multiple addresses to warn interested
parties that the Web site is in need of maintenance. You
will know that something is going wrong long before there
is any negative impact on site performance.
This article will walk you through the basic steps for
setting up ClusterCATS in a Windows NT production environment.
You will learn the basics of configuring the Domain Naming
Service (DNS) system for clustering and the required steps
for setting up ClusterCATS in a Windows NT 4.0 SP 5 and
6 (using IIS 4.0) environment with ColdFusion 4.5.1 SP 1
and 2. For additional platforms, links will be provided
to the relevant Knowledgebase (KB) articles at the end.
Configuring DNS for ClusterCATS
If you are going to add a second Web server to your existing
site, the place to start is DNS name resolution. Use DNS
round robin (RR) to distribute inbound sessions equally
between the two servers. RR for two Web servers consists
of two forward resolution (A-records) entries using the
site name. Both RR entries must resolve to each of your
fully qualified and clustered Web servers. An example follows:
Web Server1
www1.company.com 192.168.64.1 (forward entry)
192.168.64.1 www1.company.com (in-addr.arpa or reverse
entry)
Web Server2
www2.company.com 192.168.64.2 (forward entry)
192.168.64.2 www2.company.com (in-addr.arpa or reverse
entry)
Round Robin Entries
www.company.com 192.168.64.1 (forward entry)
www.company.com 192.168.64.2 (forward entry)
The inverse address resolution entries (in-addr.arpa or
reverse entry pointer records) are only used for the actual
Web servers rather than for the RR entries. The RR entries
are only forward entries (A-records). The sessions in this
example would all come in on the RR name www.company.com.
RR distributes the inbound sessions; it shares them between
www1.company.com and www2.company.com by translating
or resolving www.company.com alternatively to 192.168.64.1
and 192.168.64.2.
If you set this up in DNS and use the nslookup tool to
examine www.company.com multiple times, you will
see that it rotates between 192.168.64.1 and 192.168.64.2.
This creates a fertile bed for clustering the fully qualified
servers (www1.company.com and www2.company.com)
by eliminating the bottleneck of a single Web server. If
www.company.com were not a RR name but instead a server
name, then all inbound cluster sessions would have to go
through the chokepoint or bottleneck of a single server.
(Note: Setting up a dynamic cluster on NT requires one additional
address per server.)
Clustering Web Servers Running Windows NT 4.0 with ColdFusion
4.5.1
You may cluster your NT Web servers using static IP addresses
or you may use dynamic addresses (not to be confused with
DHCP). When clustered with CF 4.5.1 using static IP addresses,
CC IP failover causes an IP address conflict when the failed
server recovers. This IP conflict is cleared when the failed
server automatically reboots. Running dynamic Web site IP
addresses eliminates the IP conflict and the commensurate
reboot. It expedites IP failover and enables other features.
Allaire recommends using dynamic addresses (unless you are
clustering behind a NAT box or a hardware load-balancing
device) but supports both configurations. The following
procedure describes how to configure your Web site to run
on dynamic addresses.
To run dynamic, you need to have at least two IP addresses
and two Fully Qualified Host Names (FQHNs) for each Web
server. One of the IP addresses is what we call a maintenance
address. A maintenance address is an IP address bound to
the NIC on each Web server. This maintenance IP address
corresponds to the computer name and not to any Web site.
The following DNS RR example is the same as the previous
example but has been adjusted to show how it would look
for dynamic NT addressing:
Web Server1
www1.company.com 192.168.64.1 (forward entry)
192.168.64.1 www1.company.com (in-addr.arpa or reverse
entry)
computer1.company.com 192.168.64.3
(forward entry)
192.168.64.3 computer1.company.com (in-addr.arpa or reverse
entry)
Web Server2
www2.company.com 192.168.64.2 (forward entry)
192.168.64.2 www2.company.com (in-addr.arpa or reverse
entry)
computer2.company.com 192.168.64.4
(forward entry)
192.168.64.4 computer2.company.com (in-addr.arpa or reverse
entry)
Round Robin Entries
www.company.com 192.168.64.1 (forward entry)
www.company.com 192.168.64.2 (forward entry)
Based on the above example, the following steps will guide
you through the process of setting up your first cluster
with dynamic IP addresses. If your servers are already in
a cluster using static addresses and there is already only
one static address on each server that corresponds to both
the computer name and the Web site name, moving from static
to dynamic addressing requires additional steps not covered
here. If you are in this situation, click
here to read KB #15484, which describes the necessary
steps.
Step-by-Step Instructions for Dynamic IP Clustering
- What is a maintenance address? Which of the addresses
in the previous example are maintenance addresses? One
address corresponds to the name computer1. The other to
computer2. The IP addresses corresponding to the computer
or machine names are the maintenance addresses:
192.168.64.3 is a maintenance address on computer1.
192.168.64.4 is a maintenance address on computer2.
Make certain that any addresses you are adding to the
NICs on your Web servers have both forward and reverse
DNS entries corresponding with FQHNs. Modify DNS accordingly.
The computer name and the Web site name on each server
must each have a different IP address and a different
FQHN.
Right-Click on Network Neighborhood and go to Properties
- Protocols - TCP/IP - Advanced to verify or add IP
addresses. On Web Server1 (or computer1), this example
uses 192.168.64.3 and 192.168.64.1, and on Web Server2
(or computer2) we will use 192.168.64.2 and 192.168.64.4.
Make sure that the two correct addresses are bound to
each NIC on each server. Eventually the Web site address
will be unbound (dynamic). However, for now, simply bind
it to the NIC.
- In the Protocols tab, check to see if the Streams Environment
is installed. If it is not, then click add Streams Environment
to bind the new protocol.
- Go to the Computer Identification tab (the first tab
you see when you right-click on network neighborhood and
scroll to properties) and verify that the computer name
or machine name is mapped to an IP address. Note: The
computer name on the identification tab should only be
a NetBios name, not a FQHN. For example, computer1.company.com
is the FQHN the first portion of this FQHN. Computer1
can be a NetBios name, and in this case, computer1 would
appear as the computer name on the identification tab,
computer1 would also appear as the host name under the
DNS tab in protocols (TCP/IP), and the domain under the
DNS tab in this case would be company.com. The NT domain
name on the system identification tab is different; it
has nothing to do with DNS but only corresponds to your
NT domain. If the computer name does not correspond
to an IP address other than the one hosting your
Web site (if your Web site and computer both have the
same name), then refer to KB
#15484.
- Close networking. If you installed Streams Environment
or changed or added any IP addresses, reboot the servers.
- Make certain that the Streams Environment protocol is
started and set to start automatically: Start - Settings
- Control-Panel - Devices - Streams Environment - Startup
- Automatic - Start. Note that Streams Environment
is managed under devices even though it is installed as
a protocol.
- Go into the MMC (Internet Service Manager): Start
- Programs - NT4.0 Option Pack - MIIS - ISM. In the
MMC, unless you are using the default Web site, add your
virtual server by right-clicking on the top level server
and choosing New and Web Site. After you have created
a new virtual server, right-click on it and choose Properties.
Go to the advanced tab and choose Add. Make certain that
the virtual server Web site address reflects the address
of the actual Web site. These will become dynamic addresses
(not bound to the NIC), but they are still static (bound
to the NIC) and will appear on a drop-down list.
- Following our example, in IIS, you must bind 192.168.64.1
for on Web Server1, and you must bind 192.168.64.2 on
Web Server2. If the addresses were already dynamic (deleted
from the server NIC), they will not appear on the drop-down
list, and you will have to add them manually. Make sure
that you do not enter that address bound to the NIC that
corresponds to the computer name. Do not bind 192.168.64.3
or 192.168.64.4 in IIS because these addresses will become
the server maintenance addresses. After binding the Web
site address in IIS, right-click on the virtual server,
go to the Properties tab, and click on the Advanced tab.
You should see that the IP address appears, and depending
on how you have the site configured, "all unassigned"
may also appear along with the IP address. Add the IP
address again if it is missing. Since they are currently
static addresses, you may start the new virtual servers.
Note: If they were already dynamic, you will not be able
to start the virtual servers yet.
- If you have not already done so, install ColdFusion
Enterprise Server 4.5.1 SP1 or later, choosing the options
of Load Balancing and High Availability or Failover. These
options will install Allaire ClusterCATS. Reboot the servers
if prompted.
- Along with ClusterCATS, you have installed some additional
tools that are helpful to confirm and troubleshoot your
configuration. Use these command-prompt driven tools and
look for any error messages in their feedback:
Use hostinfo to verify DNS name resolution on Web Server1:
c:>cfusion\brighttiger\program> hostinfo 192.168.64.1
c:> cfusion\brighttiger\program> hostinfo www1.company.com
c:>cfusion\brighttiger\program> hostinfo 192.168.64.3
c:> cfusion\brighttiger\program> hostinfo computer1.company.com
Use hostinfo to verify DNS name resolution on Web Server2:
c:>cfusion\brighttiger\program> hostinfo 192.168.64.2
c:> cfusion\brighttiger\program> hostinfo www2.company.com
c:>cfusion\brighttiger\program> hostinfo 192.168.64.4
c:> cfusion\brighttiger\program> hostinfo computer2.company.com
Use btcfgchk to verify your configuration:
c:>cfusion\brighttiger\program> btcfgchk
Use ping to make sure that the servers can talk to each other:
c:> ping www1.company.com
c:> ping www2.company.com
c:> ping computer1.company.com
c:> ping computer2.company.com
If name resolution is correct, go on to cluster creation.
If name resolution is not correct (if the btcfgchk script
spits out any errors), do not go on until the naming
issues are resolved. Improper name resolution is the
primary source of installation and setup problems.
- You will now create the cluster using the Cluster Creation
Wizard. When prompted, do not enter the maintenance address.
You would only enter the maintenance address into the
appropriate field during cluster creation if the Web site
address (192.168.64.1 on Web Server1 and 192.168.64.2
on Web Server2) were already dynamic (i.e., not bound
to the NIC).
Following our example, we have left the Web site addresses
static. Fear not! They will become dynamic later. On
Web Server1 only, create the cluster using the Cluster
Creation Wizard (Start - Programs - ClusterCATS Explorer)
when prompted, and enter the cluster name (e.g., cat1).
(Remember that the license key is case sensitive: GoColdFusion).
Next, enter the FQHN for the first server when prompted
by the wizard (i.e., www1.company.com). The servers
will come up in Active Mode. Enter the FQHN for the
second server in the cluster (i.e., www2.company.com).
Click Finish. You have just created a cluster!
- To make the site dynamic, unbind the Web site addresses
from the server's NIC by right-clicking on Network Neighborhood
and going to Properties - Protocols - TCP/IP - Advanced.
Remove the IP address(es) corresponding to the Web site
(192.168.64.1 on Web Server1 and 192.168.64.2 on Web Server2).
Reboot both servers simultaneously. Warning: If you reboot
the servers one at a time, they will failover one to the
other. Please make sure to reboot the servers simultaneously
- You may now wish to test failover by rebooting either
server and trying to hit it with a browser while it is
down. Upon recovery, there will be no IP conflict and
no cause for a second rebooting of the failed server.
You may also wish to test redirection or load balancing
by putting one of the servers in the cluster into the
"busy" state and trying to hit that busy server through
a browser.
- Now you may set up the other features that ClusterCATS
provides, including alarms and probes.
- To set up a CFProbe, right-click on a server in CC Explorer,
choose New Monitor, and click OK. Select the blue arrow
over the label Probe Type, left-click in the Startup Parameters
field and use your keyboard arrow to move the curser to
the right without deleting any text. Change the word NORESTART
to RESTART, click on Register - Close - Close.
You now have a monitor (CFProbe) that is looking at a
ColdFusion Web page looking for the word "hello". If the
page becomes unavailable, the probe will restart the ColdFusion
server.
- To set up alarms, right-click on the cluster in CC Explorer
and choose Configure and Alarm Notification. Enter your
mail server address in the SMTP host field and type in
the e-mail addresses of anyone you want alerted for the
various events.
- To set up support mail, right-click on the cluster in
CC Explorer and choose Configure and Support. Enter your
mail server address in the SMTP gateway field and type
in the e-mail addresses of anyone you want to receive
daily reports.
- Once a cluster has been created using CC Explorer, you
can view the cluster by simply launching the CC Explorer.
Cancel the cluster creation wizard, right-click on the
teal wheel in the upper left corner the explorer and choose
View Cluster.
- To manage your cluster remotely using a Windows 98,
Windows ME, NT 4.0, or Windows 2000 laptop, simply download
the latest ClusterCATS build from the Allaire
FTP site. Note: If you install this ClusterCATS kit
on a NT or Windows 2000 Server, be sure to choose the
explorer and documentation options only during installation.
Otherwise, it will add superfluous overhead. If you wish
to manage the cluster using an explorer from outside your
firewall, you will need to open ports 9123 and 9129. Note:
This explorer will also manage Solaris- or Linux-based
clusters from your Windows laptop.
Other Platforms
If you are clustering Web servers running NES or Apache,
you will need a different procedure. For Apache on either
Solaris or Linux, see KB
#16363. For NES on NT, see KB
#15520. For NES on Solaris, see KB
#15908. For secure Apache (e.g., Raven, StrongHold,
etc.), see KB
#15856. To set up a cluster in DFP mode behind a Cisco
Local Director, see KB
#15984. To set up a cluster behind a hardware load-balancing
device that does not support DFP (e.g., Alteon, BigIP, ArrowPoint,
etc.), see KB
#15972. To set up a cluster behind a NAT firewall, see
KB #15339.
For these and many other articles go to the Allaire Knowledgebase search page. Choose the search category keyword ClusterCATS
or enter the article numbers, and you will find procedures
for clustering any platform in almost every environment.
Allaire also has a plethora of clustering experts in its
support department who will be happy to assist you with
installation and configuration.
Conclusion
This article described a scenario where a small site needed
to expand on a tight budget. As an example, a detailed setup
illustration using a hypothetical pair of IIS servers in
a model cluster was provided. While this example examines
one type of perfect clustering candidate on one type of
platform, clustering should be considered at all stages
of Web site growth and transition on all platforms.
So why wait until a crisis? Now is a perfect time to start
clustering your site.