Apps on the Grid
- By Paul Korzeniowski
- January 19, 2006
The Big Idea
BREAKING THE GRIDLOCK
- Grid apps break up computational tasks and distribute the processing to a variety of systems, based on availability.
- Early grid adopters bolster claims the tech cuts IT costs and accelerates processing performance for large-scale, complex calculations and simulations.
- Hardware and software suppliers are working on commercial grid apps; standards initiatives are under way; and debate over traditional software licensing models burns ferociously.
The traditional way of writing apps that run on top of
specific operating systems, whether they reside on
desktop systems or servers, is giving way to developing
software that can run on any system on a network.
This one-app-for-all-systems approach is the primary advantage
of grid computing, which is slowly—very slowly—migrating
from global scientific and research networks to enterprise networks.
"Because they are under pressure to cap costs, corporations
are searching for ways to make better use of their application
infrastructure," says Joshua Greenbaum, a principal at
Enterprise Applications Consulting.
Apps, such as CRM, ERP and others, tend to have processing
peaks and valleys, when systems may sometimes operate at
only 5 percent of their capacity. Grid computing tries to take advantage
of those lulls by moving processing chores from overburdened
to underutilized systems. The payoff is lower software
costs and being able to rely on fewer systems to run them.
Although the concept seems simple, deploying grid computing
can be difficult. Apps must be decoupled from their respective
hardware; middleware developed; and management tools
fine-tuned for this new environment. Because many of those
tools are not available, grid computing in enterprise environments
Some observers expect grid's acceptance to increase dramatically
in the coming years. A variety of suppliers, including Appistry,
Datasynapse, Entropia, Globus, Parabom Computation,
Platform Computing, United Devices and Univa, are developing
grid software, Mainstream hardware and software vendors
also have gotten into the market.
Few enterprise applications
So far, grid tech has been a good fit for only select applications,
mainly in the research and scientific communities. The U.S. Environmental
Protection Agency runs an application that examines
air quality measurements and produces reports that outline how well different states are doing in their efforts
to comply with federal regulations.
"We needed an application infrastructure
that was inexpensive but could
process a large volume of information,"
explains Gary Walter, networking and information
technology research and development
program manager. The EPA
built its app on Linux computers and
open-source grid app development tools,
such as the Globus toolkit and SDSC
Storage Resource Broker.
Pharmaceutical companies are using
grid computing to identify new drug
treatments. Entelos, a biotechnology
firm, relies heavily on modeling to determine
how its products will perform in
combating diseases such as diabetes. "A
drug trial can involve more than 10,000
simulations and could take up to 2 years
running on 1 server, says Alex Bangs,
CTO. "With a grid, the simulation can
be performed in less than a week." Entelos'
grid, which supports 180 processors
and relies on Platform Computing's
software, has increased the company's
processing power tenfold since 2002.
Financial enterprises are building grids
to address complicated computational
needs. They run tasks such as Monte Carlo
simulations (which randomly generate
values in spreadsheets) and perform complex
calculations that have different input
parameters, then produce financial
performance snapshots for individual and
New licensing model next step for grid
The advent of grid computing is forcing software suppliers to take a close look at their licensing models, a step not all of them are willing to take. Traditionally, software licenses have been sold on a system basis: a company has to purchase one license for each computer running the software. That model made sense when computing chores were limited to specific systems.
Grid computing introduces a different paradigm. Rather than one system, grid computing divides processing chores among multiple processors, sometimes dozens or even hundreds. In some cases, companies are not sure which systems may be available to process a grid application until it actually runs. To adhere to traditional software licensing policies, a company would be required to purchase licenses for all of its systems, something that is not practical in many cases. “Software licensing has become a hot topic for customers as well as vendors,” says Joshua Greenbaum, a principal at Enterprise Applications Consulting.
A few vendors have altered their licensing agreements, for instance, Milliman offers a version of its MG-ALFA life insurance actuarial software for grid environments, and Fluent developed a version of its computational fluid dynamics for Platform Computing’s grid infrastructure. However, these vendors are exceptions.
Suppliers fear, perhaps legitimately, that per-user or usage-based pricing, where customers pay fees based on how often they use the software, will lower revenue. They point out customers can recover the cost of any additional licenses through the savings that come from not having to purchase expensive servers. Consequently, many suppliers are sticking with CPU-based pricing models.
The Hartford Financial Services
Group, the largest seller of variable annuities
in the world, has a client base of
almost 2 million. The company wanted
to let customers examine the performance
of their annuities in real time. In the
fall of 2004, the firm moved to a grid architecture
with 200 servers running Linux,
supported by Condor, open-source
grid software developed by the University
"In some cases, we have been able to
reduce our processing time from 10
hours to under 20 minutes,'' explains
Chris Brown, director of advanced technologies.
"The ability to produce calculations
in such a short time has made our
service not only more efficient but also
more attractive to customers."
Hewitt Associates, a human resources
consulting and outsourcing firm, had a
similar requirement. The performance
of a mainframe app to calculate pension
benefits began to slow. The cost of upgrading
the system was high, so the firm
offloaded it onto a grid of Linux servers
managed by Datasynapse software. "By
moving to the grid, we have been able to
reduce our ITcosts by millions of dollars,"
states Dan Kaberon, director of computer
Users get tough
In some cases, users are playing hardball. “We told one software vendor that if it did not alter its pricing, we would go with an open-source solution,” says Gary Walter, networking and information technology research and development program manager at the Environmental Protection Agency. That threat resulted in the vendor’s lowering pricing by more than 50 percent.
Unfortunately, the Tony Soprano approach doesn’t always work. Hewitt Associates had an app it wanted to move to a grid environment, but the vendor was so inflexible about changing its system pricing model that the client killed the project, according to Dan Kaberon, director of computer resource management.
Suppliers are not expected to remain so steadfast for much longer. In addition to grid, technical advances, such as multi-core processors, storage virtualization, on-demand computing and utility computing, are putting pressure on traditional software pricing models. “Increasingly, vendors are becoming much more willing to work with customers to develop pricing metrics that make sense for both parties,” explains Frank E. Gillett, principal analyst at Forrester Research. “One reason is the vendors realize if they don’t work with the customer, a competitor will.”
Major rewrites, minor
Although many early adopters report
success with their grids, building apps on
this infrastructure can mean a lot of
work. Most apps have to be rewritten to
run on a grid. "The trend recently with
software development is to make applications
more modular, which helps companies
that want to move to grid," says
Wolfgang Gentzsch, who is working on
a grid project for the Microelectronics
Center of North Carolina.
Grid can be less complex running internally
developed apps, but what happens
when a company wishes to run third-party
software? The choices are few. New Energy
Associates, a Siemens company that
provides custom software development
and consulting services to energy companies,
adapted its software to grid. "Our applications,
which include energy market
and transmission analysis, energy trading,
risk management, and retail and wholesale
energy market operation, are complex and
processor intensive, so moving them to
grid made sense for us," says Neal Tisdale,
VPof software development.
Grid standards begin to take shape
A growing number of companies are taking a closer look at grid computing. The ability to manage workloads, monitor usage and charge for resources allocated, and enact comprehensive security capabilities are all important considerations for corporations as they begin to examine deploying a grid application. Because grid computing is still young, most products are based on proprietary technology, and therefore many corporations find it difficult to include all of the desired features in potential grid applications.
To alleviate the problem, users have been pushing suppliers toward supporting industry standards. At the moment, the work is a bit scattershot, but there are signs that vendors may start to coalesce around select standards.
The Global Grid Forum has been working on the Open Grid Services Architecture—a set of specs to make it easier for compute-intensive apps to take advantage of grid computing—and the Open Grid Services Infrastructure—the interfaces and services available to OGSA. The GGF wants to align OGSI with emerging Web services and has been melding OGSI concepts into the Web Services Resource Framework, which is overseen by OASIS.
In the spring of 2004, a vendor consortium that has EMC, Fujitsu, HP, Intel, Siemens, and Sun among its backers, launched the Enterprise Grid Alliance. The organization’s focus is on creating standards for commercial grid apps. The group developed a reference model that features a lexicon of grid terms, a model that classifies the management and lifecycles of the components required for enterprise grids, and a set of use cases that demonstrate the requirements for enterprise grid computing.
The open-source community also has moved toward developing grid products. Companies such as HP, IBM and SAP support open-source software through The Globus Alliance. The alliance’s Globus toolkit helps companies design and maintain grid apps.
The result had been a confusing hodgepodge of specifications and initiatives, but that may change. “Recently, there has been talk about merging the GGF and the EGA, and that step would mean there is a greater likelihood that one set of grid standards would emerge to help users deploy these applications,” notes Frank E. Gillett, principal analyst at Forrester Research.
Grid has not made a lot of sense for
other software vendors. "Few off-theshelf
applications are grid ready," says
Frank Gillett, principal analyst at Forrester
Research. Some vendors are leery
of changing their software to run on a
grid because that architecture may not
be as lucrative as traditional systems.
Many corporations avoid building
their own apps because it's expensive and
time consuming. Companies have turned
to internally developed grid systems only
when the app promises dramatic business
benefits, such as the case with The Hartford
Financial Services Group, or when
the app is not available from a third party,
which is what Entelos experienced.
Not a fine wine yet
Firms that do end up writing grid apps often
find such development challenging.
Currently, the grid software infrastructure—development tools, management
systems—is immature. Part of the problem
is the technology's novelty, and another
reason is a lack of standard interfaces,
a problem that vendors are moving
Grid computing means chopping up
large compute problems into manageable
bites and distributing them to a variety
of systems. Once the processing has
been completed, the infrastructure software
must be able to put all of the pieces
back together—a task that becomes
even more challenging when the machines
are not running at the same time
or working with the same operating systems.
Grid software suppliers have been
working to add workload management
features, which help move information
among computers, and failover response
functions, which make sure that important
transactions are completed.
In addition, extensions are being
made to system management tools,
such as Computer Associates' Unicenter
and IBM's Tivoli, to accommodate
Putting procedures in place
Grid also introduces personnel challenges.
Most grid applications take processing
cycles from servers, but grid
proponents have been pushing companies
to capture unused desktop and laptop
computing cycles. While this could
help companies reduce their IT costs,
it requires that many put new policies
and procedures in place. Many large
companies do not have a good grasp on
how many computers are in their organization
and have to begin with a
comprehensive inventory. Once they
comple te that task, they need to chart
system usage patterns and deduce how
much unused processing power may be
available to different applications.
Identifying unused processing power is
one thing, getting departments to share
underutilized resources is another. Corporations
tend to operate in stand-alone
fashion, with departments responsible for
specific applications, but that structure
changes in new consolidated grid environments. "Someone has to be in charge
of determining when to take cycles from
one machine or another," says Enterprise
Applications Consulting's Greenbaum.
"Some managers may not want to let go
and share their resources because they
feel they'll lose control over their computers,
lose part of their budget or lose
control of their workflow."
Entelos experimented with incorporating
desktops as part of its grid, but removed
them because they added problems
instead of horsepower. Bangs explains, "Grid does not work well with laptop
users, unless they leave their laptops in the
office all of the time, which usually isn't the
case. Currently, grid software does not do
a good job in recognizing when someone
is dialing in from a remote location and figuring
out a way to use that system."
While grid technology has potential,
its long-term future is unclear. "Grid isn't
well suited to applications that require
a lot of updating," asserts Hewitt
Associates' Kaberon, who thinks the
technology will remain a niche rather
than a mainstream application development
option. For example, in many
ERP and CRM systems, one action is
dependent upon the result of a previous
one, so transactions, such as selling
a specific airline seat, can not be broken
into several tasks and distributed
across a grid.
Others are more optimistic about
grid's future. "If the movement in software
development is away from singlethreaded
to multi-threaded applications
and away from hierarchical to component-
based application development,
then a growing number of applications
should become more suited to running
on grids," concludes Forrester Research's
Gillett. "If those assumptions are correct,
the question then becomes: How long
will the migration to grid take? Judging
from past experiences—a lot of corporate
data is still stored in relational database
management systems—the software
industry is looking at a long-term rather
than an immediate change."
ILLUSTRATION BY TODD DAVIDSON