 |
|
|
|
|
|
|
|
Clustering
Multiple NES Instances on Solaris
Frank DeRienzo, MBA, MCSE
Principal Technical Support Engineer
Macromedia, Inc.
Clustering on Solaris is similar to clustering on NT or
Linux. Successful clustering on each platform depends on
correctly configuring the operating system, name resolution,
and the Web server. The following article will explain the
steps necessary to set up multiple clusters using different
NES instances on Solaris.
This seven-part article is example-based, but because there
are so many supported flavors of Solaris, the article begins
by describing required modifications to run on each of the
various supported versions. Parts two and three of the article
discuss configuring the IP stack and installing and configuring
the NES Web server. Part four describes how to add a hardware
virtual server under an NES instance. Part five covers the
installation of ColdFusion (CF) and ClusterCATS (CC) to
the first instance. Part six adds and configures a second
NES instance. Part seven shows how to configure multi-instance
servers into clusters. If you follow the steps in this example-based
article, you will successfully configure multiple NES instances
with ColdFusion (CF) clusters.
Part 1: Basic System Requirements and Required Patches
ColdFusion 4.5.1 SP2 can be clustered on Solaris 2.51
running on SPARC as long as you update the OS with the following
four patches:
- Solaris patch 103640-xx (kernel patch)
- Solaris patch 103663-xx (libresolv.so for threaded
applications)
- Solaris patch 106529-xx (LibC: Shared library patch
for C++)
- Solaris patch 103582-xx (/kernel/drv/tcp patch)
Important Note: If you are running CF on Solaris 2.5.1,
please note that CF 5.0 will not support Solaris 2.5.1.
A CF cluster will run on Solaris 2.6 with the following
two patches:
- Solaris patch 105181-xx (kernel patch)
- Solaris patch 105591-xx (LibC: Shared library patch
for C++)
For Solaris 7 you will need Solaris patch 106327-xx
(LibC: Shared library patch for C++)
Download and install the following Solaris package: SUNWxcu4
-- The XCU4 Utilities
These patches and packages are all available for download
at http://access1.sun.com.
To view the history of patches already installed on your
Solaris system, go to the /usr/sbin directory:
/usr/sbin patchadd -p
After bringing the operating system up to date, examine the
hardware to be clustered. For a growing site where purchasing
new hardware can be cost prohibitive, clustering can help
postpone the need for a major hardware upgrade by maximizing
the potential of available platforms combining them to emulate
as one server. With that said, clustering does add some overhead
to a system. Clustering on a server with 128 MB RAM is possible,
but a minimum of 256 MB RAM is recommended. The same numbers
apply to available disk space. While the Web content on clustered
systems must be mirrored across all clustered systems, you
do not have to cluster like systems; for instance, you may
cluster a slow system with a fast system, with a mid-range
system, etc. to maximize the potential of available machines.
Note that ClusterCATS does not support running different types
of Web servers on the same cluster. If a Solaris system is
configured to run both NES & Apache, you can choose to configure
ClusterCATS with only one of these Web servers. This article
focuses on clustering Netscape Enterprise Server (NSAPI) 3.51
or higher on Solaris, but Apache (1.3.6 or later) can be clustered
on either Solaris or Linux.
For information on clustering Apache servers, see
Configuring ClusterCATS Failover on Apache Web-Servers.
Part 2: Configuring the Solaris Operating System: IP Stack
and Name Resolution
Now that the operating system and hardware are updated, focus
on the IP stack and name resolution. NIS can be problematic
as can any form of name resolution if it is not set up properly.
ClusterCATS requires that the group btcats exist. It will
attempt to create this group during the ColdFusion installation
using groupadd. If using NIS or NIS+, make sure that either
nsswitch.conf allows for group resolution from the group file,
or that the group btcats gets created in NIS/NIS+ prior to
installing ClusterCATS. It is always a good idea to install
with root access privileges.
Most of the local name resolution work on a Solaris system
is done in the /etc directory. The first file to inspect
(cat) and -- if needed -- edit is: /etc/nsswitch.conf
vi /etc/nsswitch.conf
Add dns as needed to the hosts line in /etc/nsswitch.conf:
# /etc/nsswitch.conf fragment:
| -passwd: |
files |
| -group: |
files |
| -hosts: |
files dns |
| -networks: |
files |
| -protocols: |
files |
| -rpc: |
files |
Note: ClusterCATS requires correct name resolution. You can
use either hosts files, dns or NIS, but whichever you choose
to use, it must be correctly configured. In the nsswitch.conf
file pay particular attention to the hosts entry. If you are
using dns, make certain that dns is listed first to speed
up name resolution. If you are relying primarily on a hosts
file for internal private address resolution (behind a hardware
load balancing device or NAT box), then place files before
dns.
The next file to inspect or create is: /etc/resolv.conf
#vi resolv.conf
nameserver 192.168.0.172
nameserver 192.168.0.214
domain dev.nashua.allaire.com search allaire.com qa.nashua.allaire.com
dev.nashua.allaire.com
Make certain that the resolv.conf file points the Web server
to the correct dns servers and searches the correct domains.
The IP addresses above simulate dns servers.
It may be necessary to specify a default gateway and default
domain on the server. Do so by creating: /etc/defaultrouter
& /etc/defaultdomain files
#vi defaultrouter
192.168.0.1
#vi defaultdomain
dev.nashua.allaire.com
ClusterCATS Cluster Members must each have a unique IP address.
The name used to cluster a Web server, or virtual server must
be the fully qualified host name; it must be the name that
forward and reverse maps to the IP address of the Web server
that is being clustered. This is not a Round-Robin name. This
is not the name of a software virtual server; NES software
virtual servers are not supported neither are IIS Virtual
Host Headers, nor Software Virtual Servers on Apache. ClusterCATS
cannot cluster multiple Web sites (multiple host names) on
a single IP address under any circumstances. If you must cluster
multiple host names on a single IP address, the best way to
do it is with a high-end hardware load-balancing device such
as Cisco's Local Director, Content Services Switch, Alteon's
ACEdirector, F-5's BIGIP, etc.
Part 3: Configuring the NES Web Server for Clustering
Let's start with basics: unzip, extract, and install NES (IPlanet)
on a server called anselm.dev.nashua.allaire.com. The following
command will do it all on one line:
gzip -dc enterprise-4.0SP2-export-us.sparc-sun-solaris2.6.tar.gz
| tar xvf - ./setup /usr/netscape/server4/https-anselm.dev.nashua.allaire.com
After installation, manually start the NES admin server prior
to administering the site:
/usr/netscape/server4/https-admserv ./start
After starting the admin server, the URL to configure NES
instances and hardware virtual servers on anselm.dev.nashua.allaire.com
will be:
http://anselm.dev.nashua.allaire.com:8888/admin-serv/bin/index
Install NetScape Navigator if it is not already on the Web
server. Do not try to administer NES from an IE browser. Launch
Netscape browser instead and administer Netscape Server. Create
an NES instance, and if you would like, you may also create
hardware-virtual servers under that instance; multiple Web
sites do not require multiple instances. Remember throughout
this procedure to: HIT APPLY - APPLY YOUR CHANGES! Failure
to apply your changes will cause loss of those changes. Later
in this procedure it could include a loss of configured software
in the NES obj.conf file; ClusterCATS and ColdFusion entries
could be overwritten with a blank (default) obj.conf.
The IP address of this instance is 192.168.64.160. This
is in anticipation of additional instances or of hardware
virtual servers added under the instance; multiple clustered
servers require binding specific IP addresses. If you are
only clustering one instance and have no additional Web
servers to configure, leave this set to the default inaddrany
IP address.
Part 4: Adding a Hardware Virtual Server (Optional Step)
Do you need an additional Web site without the overhead and
complexity of an additional instance? If so, a quick way to
add one is by configuring a hardware virtual server. Feel
free to skip this step if you are only interested in setting
up two instances to support both secure and non-secure traffic.
To add a hardware virtual server under this instance, first
add a registered IP address to the NIC of the Solaris server.
This takes us back to the DNS server and back into the /etc
directory. First register the new IP address and hostname
in DNS, then create the file hostname.hme0:01 in the /etc
directory and enter the new name (associated in DNS with the
new IP address). Let's create a hardware virtual under anselm
called apriori:
vi /etc/hostname.hme0:01
apriori
Next edit the hosts file by adding the new IP address and
host name.
vi /etc/hosts
192.168.64.109 apriori.dev.nashua.allaire.com anselmvs1
After laying this foundation, reboot the server to ping the
new hostname.
Now you are ready to add a hardware virtual server. First,
make sure to apply the changes. If changes were not applied
when you initially created the instance, and again when
you add the new hardware virtual server to the instance,
you will overwrite the obj.conf file. Later this will eliminate
ColdFusion and ClusterCats from the filter list. The NES
administrator will warn you before it overwrites the file.
Part 5: Installing ColdFusion With ClusterCATS to the
First NES Instance
At this point, we have configured one single NES instance
with one hardware virtual server under that instance. Doing
this on two servers, essentially allows you to run two Web
sites clustered across two platforms, but multiple instances
are required on NES in order to run both secure and non-secure
sites. NES requires that any single instance be exclusively
secure or non-secure. Additionally, ColdFusion 4.5.1 will
only recognize one instance during the installation. Let’s
go on through the installation of CF. Starting the same way
we did with the installation of the gzipped NES kit:
# gzip -dc coldfusion451Mar22.tar.gz | tar xvf
-
x README, 6652 bytes, 13 tape blocks
x releasenotes.htm, 68952 bytes, 135 tape blocks
x new.htm, 92142 bytes, 180 tape blocks
x coldfusion451Mar22.pkg, 157555712 bytes, 307726 tape blocks
x cfregup.sh, 3814 bytes, 8 tape blocks
x README.cfregup, 2399 bytes, 5 tape blocks
Next use the package-add command to install CF; make sure
you also add CC.
# pkgadd -d coldfusion451Mar22.pkg
Do you want to install ClusterCATS for ColdFusion [y]?
Do you want this server to provide failover services [y]?
The instance-name can cause a CF installation error if you
used the fully qualified hostname to create the NES instance
directory structure as recommended. It is easy to navigate
around the error:
- Enter the Netscape installation directory:
/usr/netscape/server4/https-anselm.dev.nashua.allaire.com
ERROR: Unable to find Netscape 'config' directory.
This is the directory which contains the files: obj.conf,
magnus.conf and mime.types. It is usually found in a
directory such as /usr/netscape/suitespot/https-surf/config,
where /usr/netscape/suitespot' is the Netscape installation
directory entered above and 'surf' is the hostname of
the Web server.
- Enter the Netscape 'config' directory:
/usr/netscape/server4/https-anselm.dev.nashua.allaire.com
/config
The second time you enter the string, the installation script
will accept it. During installation of the Web server, it
is still best to use the fully qualified hostname to create
the NES instance and its commensurate directory. If you go
into that directory after a proper installation, you will
be able to cat the obj.conf file and see ClusterCATS in the
filter: # /usr/netscape/server4/https-anselm.dev.nashua.allaire.com/config
# cat obj.conf Init fn=load-modules shlib="/usr/lib/btcats/program/teserver.so"
funcs="btcats_server_init,btcats_nsapi_AuthTrans,btcats_nsapi_NameTrans,btcats_ErrorFixup"
Init fnInit fn="btcats_server_init"…...
NameTrans fn=btcats_nsapi_NameTrans
AuthTrans fn=btcats_nsapi_AuthTrans
Error fn=btcats_ErrorFixup reason="BrightTiger"
Service fn="DoCFRequest" method="(GET|POST)" type="magnus-internal/cold-fusion"
NameTrans fn="pfx2dir" from="/btauxdir"
dir="/usr/lib/btcats/btauxdir"
You will also find the magnus.conf file in the config directory
of the instance. If you are only clustering one pair of
server (one instance on each server) then you will leave
the magnus.conf address entry set to inaddrany, but if you
are planning on having multiple clusters, you must put in
the IP address which corresponds to the name of the instance:
#cat magnus.conf
ServerID https-anselm.dev.nashua.allaire.com
ServerName anselm.dev.nashua.allaire.com
Port 80…..
RqThrottle 512
StackSize 131072
Address 192.168.64.160
Part 6: Configuring ColdFusion and ClusterCATS to Recognize
a Second NES Instance
If you haven’t already done so, add a second NES instance
on a separate IP address through the same browser interface
that you used to create your first instance. Make sure that
your separate IP address is bound to the NIC and registered
in DNS. Choose the add server button and remember to apply
your changes:
http://anselm.dev.nashua.allaire.com:8888/admin-serv/bin/index.
When you are setting up multiple NES instances, you will
notice that a separate directory is created for each server
instance. In each separate instance directory there will
be start and stop commands. Each instance needs its own
unique name and IP address. Here is an example of how two
NES instances should appear at your command-line. Notice
that I am still using the fully qualified names:
/usr/netscape/server4/https-anselm.dev.nashua.allaire.com
/usr/netscape/server4/https-secur-anselm.dev.nashua.allaire.com
Note: Do not confuse the second instance with the additional
hardware virtual server we added under the first instance.
A hardware virtual server is dependent upon an instance;
it sits under an instance and lives and dies with its parent
instance. A second instance is independent. If one instance
stalls, a second instance on the same system is not necessarily
affected. Running multiple instances is more resilient than
running one instance.
Also note: Netscape Enterprise Server in secure mode,
and secure Apache servers, require a keyfile password
to be started. The Allaire ClusterCATS installation script
will not restart the Netscape Enterprise Server or Apache
server during installation.
If the Web server is running with security enabled.
You may start the Web server after the installation using
your preferred method.
The ClusterCATS Web server monitor will not be able
to restart a Web server with keyfile passwords on startup.
See the following URL for other means: http://help.netscape.com/kb/corporate/19970506-1.html.
Now that we have installed CF with CC, let's configure
multiple instances. CF 4.5.1 will only recognize one instance
during the installation. After the installation, the second
NES instance must be manually configured for both CF&
CC. After Installing CF4.5.1 and adding the new instance
you must edit the 'mime.types' file to add a new ColdFusion
type.
#/usr/netscape/server4/https-secur-anselm.dev.nashua.allaire.com
/config/
# vi mime.types
Add the line:
type=magnus-internal/cold-fusion exts=cfm,dbm
Edit the 'obj.conf' file to add new service and init directives:
# /usr/netscape/server4/https-secur-anselm.dev.nashua.allaire.com
/config/
#vi obj.conf
Add the following all on one line:
Init fn="load-modules"shlib="/usr/netscape/server4/plugins/coldfusion/coldfusion.so"funcs="DoCFRequest"
Add the following line in the "default" Object:
Service fn="DoCFRequest" method="(GET|POST)"
type="magnus-internal/cold-fusion"
Stop and re-start the Web server on both platforms.
Note: If you run the Netscape Server Manager, click
on the "Apply" button to reload the new configuration
files. ColdFusion now sees your new instance.
So far we have been only working on one Solaris platform.
To go on to clustering, you will need a second mirrored
Solaris platform, so let’s expand our example according
to the diagram below and make a cluster:
 Now let's get ClusterCATS to see multiple instances
then get the servers clustered. Obviously a cluster requires
at least two platforms. We have been working on the example
platform on the left side of the diagram above called
anselm.dev.nashua.allaire.com. We installed ColdFusion
to the NES instance called anselm.dev.nashua.allaire.com.
We then added an optional hardware virtual server under
anselm.dev.nashua.allaire.com called apriori.dev.nashua.allaire.com
(not depicted in the diagram). Next we added a secure
NES instance: secur-anselm.dev.nashua.allaire.com. We
did all this to a single Solaris server. Now we need an
additional mirrored Solaris server running multiple NES
instances to create a cluster. The platform on the right
of the diagram has an instance called aquinas.dev.nashua.allaire.com;
anselm and aquinas represent mirrored sites and will be
clustered together. Likewise, secur-anselm.dev.nashua.allaire.com
and secur-aquinas.dev.nashua.allaire.com are mirrored
and will be in their own separate cluster. The hosts file
on each platform looks like this:
vi /etc/hosts
192.168.64.160 anselm.dev.nashua.allaire.com anselm
192.168.64.170 secur-anselm.dev.nashua.allaire.com secur-anselm
192.168.64.161 aquinas.dev.nashua.allaire.com aquinas
192.168.64.171 secur-aquinas.dev.nashua.allaire.com
secur-aquinas
Note: If you added a hardware virtual server to each unsecure
instance, then the hosts file would look like this:
192.168.64.160 anselm.dev.nashua.allaire.com
anselm
192.168.64.109 apriori.dev.nashua.allaire.com apriori
192.168.64.170 secur-anselm.dev.nashua.allaire.com secur-anselm
192.168.64.161 aquinas.dev.nashua.allaire.com aquinas
192.168.64.110 aposteriori.dev.nashua.allaire.com aposteriori
192.168.64.171 secur-aquinas.dev.nashua.allaire.com secur-aquinas
Let's cluster! The tools you need are in the btcats directory
on anselm:
#cd /opt/coldfusion/btcats
or if you are running CF 5.0
#cd /opt/coldfusion/cfam/scripts
Configure the initial instance by using the btadmin tool:
./btadmin config failover https-anselm.dev.nashua.allaire.com
Note: If you are running multiple instances in the passive-mode
either behind a Cisco LocalDirector or other hardware loadbalancing
device, you will not need to subset or config failover.
You will only need to add btcats to the additional instances
using the add command - outlined below. Subsetting enables
CC failover.
Also note: The name of the instance that you are placing
after the command ./btadmin config or later in this
procedure that you will place after the command ./btadmin
add should be the fully qualified long name. The same
name that appears in dns should also appear as the NES instance
name and in the host file - these names should all be the
same fully qualified name as this example indicates. Mixing
up naming conventions with multiple clustered instances
on NES will comlicate your configuration and make troubleshooting
more difficult. Try to be consistent and systematic, it
will save you trouble later.
This is where you design your cluster subset. Which other
servers will be in this cluster? Which other instances or
virtual servers belong in this cluster? Enter their names.
Following our example, you will want to cluster aquinas
with anselm:
aquinas.dev.nashua.allaire.com
Next you must go to the command prompt on aquinas and configure
it to fail-over to anselm:
#cd /opt/coldfusion/btcats
or if you are running CF 5.0
#cd /opt/coldfusion/cfam/scripts
Assuming that aquinas.dev.nashua.allaire.com was the instance
to which you installed, configure it using the btadmin tool:
./btadmin config failover https-aquinas.dev.nashua.allaire.com
This is where you complete your first cluster subset. Following
our example, you will want to match anselm with Aquinas
by entering the name anselm:
anselm.dev.nashua.allaire.com
Now let's focus on the new instances by going back to the
command prompt on anselm. Add the new instance to begin
creating your next cluster. Type in the following:
./btadmin add btcats https-secur-anselm.dev.nashua.allaire.com
The script will ask:
server type - netscape or apache - enter netscape
server root - /usr/netscape/server4/https-secur-anselm.dev.nashua.allaire.com
load-balancing options - http or ColdFusion - choose ColdFusion
Now that you have added the new instance with the btadmin
tool, you must configure it. Type in the following:
./btadmin config failover https-secur-anselm.dev.nashua.allaire.com
Once again for this new cluster, which other servers will
be in this cluster? Which other instances or virtual servers
belong in this cluster? Enter their names. Following our
example:
secur-aquinas.dev.nashua.allaire.com
Add the new instance by going to the command prompt on aquinas.
Add the new instance to begin creating your next cluster.
Type in the following: ./btadmin add btcats https-secur-aquinas.dev.nashua.allaire.com
The script will ask:
server type - netscape or apache - enter netscape
server root - /usr/netscape/server4/https-secur-aquinas.dev.nashua.allaire.com
load-balancing options - http or ColdFusion - choose ColdFusion
Now that you have added the new instance with the btadmin
tool, you must configure it. Type in the following:
./btadmin config failover https-secur-aquinas.dev.nashua.allaire.com
Once again for this new cluster, which other servers will
be in this cluster? Which other instances or virtual servers
belong in this cluster? Enter their names. Following our
example:
secur-anselm.dev.nashua.allaire.com
After subsetting is complete, upon clustering, secur-anselm
and secur-aquinas become failover partners; the NES instances
called aquinas and anselm also will be failover partners.
Before clustering, check your btinstall.config file
(in /opt/coldfusion/btcats/database or on 5.0 in /opt/coldfusion/cfam/database)
to make sure that the BT_NSBIND is not set to inaddrany.
When running multiple instances BT_NSBIND may read anything
but inaddrany. For our example, type in fred
in place of inaddrany.
Now you are ready to set up your cluster using either
the BTweb interface or the Windows-based ClusterCATS Explorer.
If you choose to use the BTweb browser-based interface,
keep in mind that it is limited insofar as it has no interface
to set up ClusterCATS to work with Cisco Local Director
with the dynamic feedback protocol. ClusterCATS integration
with Local Director requires the windows-based ClusterCATS
explorer. The explorer is an excellent GUI: if you want
to manage your cluster from a remote workstation, laptop,
etc., simply download the ClusterCATS explorer and install
it on a windows platform: ftp://ftp.allaire.com/outgoing/clustercats/ccwin2k.exe
When installing this package, choose only the explorer
option. It will manage your cluster using port 9123 (and
9129 if you use CF probes); if the explorer is on the
opposite side of a firewall from the servers, you will
need to open ports 9123 and 9129 on the firewall. If you
are running on Solaris (or Linux) and choose not to download
the explorer, you may use the Btweb browser interface
to configure your cluster. Our example follows using BTweb.
Part 7: Building the Cluster Wsing a BTWeb Browser Interface
If you are familiar with the Windows-based ClusterCATS explorer,
then the browser-based BTWeb interface will be easy for
you. If you are using Netscape Enterprise Server V3.x, the
URL for accessing the ClusterCATS Explorer Web addition
(btweb) is: http://admin-serv/btweb/default.html If you
are using Netscape Enterprise Server V4.0x, the URL for
accessing the ClusterCATS Explorer Web addition (btweb)
is: http://server-name>:/https-admserv/btweb/default.html
Following our previous example using your NES admin server
password:
http://anselm:8888/https-admserv/btweb/default.htm
This first screen is where you will create your two clusters.
If you already have created a cluster, you may view it by
choosing show cluster.
You may view the cluster level after adding your first server.
Following our example, at this screen above you would choose:
add server and enter aquinas.dev.nashua.allaire.com - the
second server in the cluster.
You may configure all the individual server attributes by
connecting to each clustered server. By following the menus
on the left side of the browser screen, you will find all
the same options offered in the Windows-based explorer:
session-awareness, probes, alarms, reporting, and so on.
Conclusion
The combination of Solaris, Netscape Enterprise Server,
ColdFusion, and ClusterCATS creates a robust Web site. If
you followed this procedure, you now have multiple NES instances
clustered across multiple servers. You may also have chosen
the option to create separate clusters containing NES hardware
virtual servers created under each instance. Macromedia
application servers are offered for all major platforms.
You choose the operating system and web-server and we will
provide the application server solution.
|
|
|
|
|
|
|
