By Jim Schley
Principal Engineer
Macromedia, Inc.
One of the most exciting new features of ColdFusion
5.0 is the introduction of the bundled Verity
K2 Server. A very common hindrance to scalability
of ColdFusion applications has historically been
the search facility. K2 eliminates that problem
by providing ColdFusion applications with a highly
scalable search engine facility based on a TCP/IP
client-server architecture. Additionally, Verity
K2 Server provides considerable performance improvement
over the basic, file-based Verity search functionality
available with earlier versions.
Background
Verity provides Macromedia with the Verity Developer's
Kit (VDK) to allow Verity search and indexing
technology to be plugged into ColdFusion. Versions
of ColdFusion prior to version 5.0 relied on the
VDK alone for functionality behind all of the
search and indexing tags such as CFSEARCH, CFCOLLECTION,
CFINDEX, etc.
Unlike K2 Server, VDK is not a client-server
technology. The VDK stores all of the information
that make up a Verity collection on the local
filesystem. Because of this, handling Verity collections
in a clustered ColdFusion server environment has
traditionally been a tricky proposition prior
to ColdFusion 5.0 because each server needed filesystem
access to the collection files. This generally
meant that - in an optimal configuration - each
server needed to maintain a separate copy of the
Verity collection files. This created the requirement
of keeping separate copies of the same collection
synchronized.
The VDK is still an integral part of ColdFusion
5.0. An upgraded version of it (2.6.1) is included,
and it still provides the core indexing technology
of the application server (CFCOLLECTION and CFINDEX).
By default, the VDK is also still used as the
search technology (CFSEARCH) for ColdFusion 5.0.
If left in this "VDK mode," ColdFusion 5.0 will
provide search functionality in a similar fashion
to previous versions.
Implementing Verity K2 Server
Verity K2 Server ships with ColdFusion 5.0, but
it is not implemented in the out-of-box configuration.
In order to take advantage of K2, ColdFusion must
be switched over to "K2 mode" from "VDK mode."
The VDK will still be used by ColdFusion after
switching to K2, but only for indexing and collection
creation; all searches will executed by the K2
search engine.
Unfortunately, it is mostly a manual configuration
process to get Verity K2 Server up and running,
but fortunately the procedure is very well documented
in Chapter 6 of the ColdFusion 5.0 Advanced Administration
manual entitled "Configuring Verity K2 Server."
This manual is available online at www.allaire.com/documents/cf50/cf5_adv_admin.pdf.
Cluster Configuration Options
The Verity K2 Server's TCP/IP communication
and client-server architecture make it ideally
suited for solving the traditional problems associated
with maintaining Verity collections in a cluster
of ColdFusion servers. There are two options that
you can choose from for this configuration depending
on your cluster specifications.
The first option is to take a ColdFusion server
and keep it out of your cluster rotation and dedicate
it for use as your search server. In this configuration,
the dedicated search server is not used to process
and serve Web pages as a member of the cluster,
but rather, its resources are dedicated to maintaining
the Verity collections and to handling the search
requests from the clustered ColdFusion servers.
The second option is to designate one server
in your existing cluster as the search provider
without taking it out of your cluster rotation.
In this configuration, your search provider will
still be used to process and serve Web pages as
a member of the cluster, but it will take on the
additional responsibility of handling all of the
search requests for all of the other servers in
the cluster.
There are pros and cons to both configurations;
however, the first option is clearly a better
option if you can dedicate resources to this function
without reducing the overall capacity of your
cluster. If the load on your cluster due to search
requests is not very significant, you may not
require dedicated resources for your search facility,
and you might benefit more by having the search
server also participate in the cluster as a member
Web server. In this case, the second option is
for you. Clearly, capacity planning and analysis
is the only way to make a solid decision. If the
search functionality of your application is a
prominent or integral feature or if you have more
than a few servers in your cluster, you should
consider dedicating a server for your Verity K2
Server to allow for the maximum scalability.
Setting up your Cluster with K2 Server
Regardless of which configuration you use, the
setup procedure for your cluster is the same.
The first step is to ensure that the files that
comprise the collections you are using in your
application reside locally on the search provider.
Then, per the Advanced Administration manual,
configure the k2server.ini file to include the
appropriate references to these collections, configure
the port that the K2 Server will run on, and then
start up the server.
The final step is to go to the Verity Server
page in the ColdFusion Administrator of each clustered
server and specify the hostname and port of the
search provider server. Once you've completed
this step, all of the cluster members will be
using the centralized search provider when CFSEARCH
is executed, and the processing of the search
logic will occur on the search server as opposed
to the local ColdFusion server instance.
It is the best practice to run all collection
maintenance activities such as re-indexing and
optimizing on the search provider machine. This
is especially advantageous when a dedicated server
is utilized in order to segregate the processing
required for these activities from the production
Web servers. For file-based Verity collections,
the files being indexed should be available locally
to the search provider. For custom collections,
the database connections should be available to
the search provider.
Some Web applications require the ability to
programmatically update collections from the application
itself. Because the application code will be making
calls to CFINDEX, this means that the VDK on each
Web server will require access to the centrally
located collection files on the search provider
and be setup as "mapped collections" locally.
This is easily accomplished by providing network
file system access to the collection files on
the search provider from the Web server. On NT,
UNC naming conventions and file sharing are sufficient
to do this, and on UNIX, NFS and Samba (SMB) are
viable solutions.
Additional Scalability, Redundancy, and Expandability
If you find that the new features available
from ColdFusion 5.0 are still not providing your
enterprise with the level of advanced scalability
that you require, seamless integration with Verity
Enterprise may be the solution you're looking
for. Verity Enterprise - available directly from
Verity - provides unlimited scalability by brokering
search requests out to multiple K2 Servers. This
architecture also allows for fault tolerance features
in addition to increased management functionality
and other added elements. For complete details
on Verity Enterprise, see Verity's web site: www.verity.com.
Integrating a Verity Enterprise configuration
with your cluster is as simple as specifying the
Verity Broker server in ColdFusion Administrator
as your Verity Server. All of the collections
in your Verity Enterprise solution will then be
available to ColdFusion for searching.
|
