Apps on the Grid -- ADTmag

Apps on the Grid

By Paul Korzeniowski
January 19, 2006

The Big Idea

BREAKING THE GRIDLOCK

Grid apps break up computational tasks and distribute the processing to a variety of systems, based on availability.
Early grid adopters bolster claims the tech cuts IT costs and accelerates processing performance for large-scale, complex calculations and simulations.
Hardware and software suppliers are working on commercial grid apps; standards initiatives are under way; and debate over traditional software licensing models burns ferociously.

The traditional way of writing apps that run on top of specific operating systems, whether they reside on desktop systems or servers, is giving way to developing software that can run on any system on a network. This one-app-for-all-systems approach is the primary advantage of grid computing, which is slowly—very slowly—migrating from global scientific and research networks to enterprise networks. "Because they are under pressure to cap costs, corporations are searching for ways to make better use of their application infrastructure," says Joshua Greenbaum, a principal at Enterprise Applications Consulting.

Apps, such as CRM, ERP and others, tend to have processing peaks and valleys, when systems may sometimes operate at only 5 percent of their capacity. Grid computing tries to take advantage of those lulls by moving processing chores from overburdened to underutilized systems. The payoff is lower software costs and being able to rely on fewer systems to run them.

Although the concept seems simple, deploying grid computing can be difficult. Apps must be decoupled from their respective hardware; middleware developed; and management tools fine-tuned for this new environment. Because many of those tools are not available, grid computing in enterprise environments is rare.

Some observers expect grid's acceptance to increase dramatically in the coming years. A variety of suppliers, including Appistry, Datasynapse, Entropia, Globus, Parabom Computation, Platform Computing, United Devices and Univa, are developing grid software, Mainstream hardware and software vendors also have gotten into the market.

Few enterprise applications
So far, grid tech has been a good fit for only select applications, mainly in the research and scientific communities. The U.S. Environmental Protection Agency runs an application that examines air quality measurements and produces reports that outline how well different states are doing in their efforts to comply with federal regulations.

"We needed an application infrastructure that was inexpensive but could process a large volume of information," explains Gary Walter, networking and information technology research and development program manager. The EPA built its app on Linux computers and open-source grid app development tools, such as the Globus toolkit and SDSC Storage Resource Broker.

Pharmaceutical companies are using grid computing to identify new drug treatments. Entelos, a biotechnology firm, relies heavily on modeling to determine how its products will perform in combating diseases such as diabetes. "A drug trial can involve more than 10,000 simulations and could take up to 2 years running on 1 server, says Alex Bangs, CTO. "With a grid, the simulation can be performed in less than a week." Entelos' grid, which supports 180 processors and relies on Platform Computing's software, has increased the company's processing power tenfold since 2002.

Financial enterprises are building grids to address complicated computational needs. They run tasks such as Monte Carlo simulations (which randomly generate values in spreadsheets) and perform complex calculations that have different input parameters, then produce financial performance snapshots for individual and institutional accounts.

New licensing model next step for grid

The advent of grid computing is forcing software suppliers to take a close look at their licensing models, a step not all of them are willing to take. Traditionally, software licenses have been sold on a system basis: a company has to purchase one license for each computer running the software. That model made sense when computing chores were limited to specific systems.

Grid computing introduces a different paradigm. Rather than one system, grid computing divides processing chores among multiple processors, sometimes dozens or even hundreds. In some cases, companies are not sure which systems may be available to process a grid application until it actually runs. To adhere to traditional software licensing policies, a company would be required to purchase licenses for all of its systems, something that is not practical in many cases. “Software licensing has become a hot topic for customers as well as vendors,” says Joshua Greenbaum, a principal at Enterprise Applications Consulting.

A few vendors have altered their licensing agreements, for instance, Milliman offers a version of its MG-ALFA life insurance actuarial software for grid environments, and Fluent developed a version of its computational fluid dynamics for Platform Computing’s grid infrastructure. However, these vendors are exceptions.

Suppliers fear, perhaps legitimately, that per-user or usage-based pricing, where customers pay fees based on how often they use the software, will lower revenue. They point out customers can recover the cost of any additional licenses through the savings that come from not having to purchase expensive servers. Consequently, many suppliers are sticking with CPU-based pricing models.

—Paul Korzeniowski

The Hartford Financial Services Group, the largest seller of variable annuities in the world, has a client base of almost 2 million. The company wanted to let customers examine the performance of their annuities in real time. In the fall of 2004, the firm moved to a grid architecture with 200 servers running Linux, supported by Condor, open-source grid software developed by the University of Wisconsin.

"In some cases, we have been able to reduce our processing time from 10 hours to under 20 minutes,'' explains Chris Brown, director of advanced technologies. "The ability to produce calculations in such a short time has made our service not only more efficient but also more attractive to customers."

Hewitt Associates, a human resources consulting and outsourcing firm, had a similar requirement. The performance of a mainframe app to calculate pension benefits began to slow. The cost of upgrading the system was high, so the firm offloaded it onto a grid of Linux servers managed by Datasynapse software. "By moving to the grid, we have been able to reduce our ITcosts by millions of dollars," states Dan Kaberon, director of computer resource management.

Users get tough

In some cases, users are playing hardball. “We told one software vendor that if it did not alter its pricing, we would go with an open-source solution,” says Gary Walter, networking and information technology research and development program manager at the Environmental Protection Agency. That threat resulted in the vendor’s lowering pricing by more than 50 percent.

Unfortunately, the Tony Soprano approach doesn’t always work. Hewitt Associates had an app it wanted to move to a grid environment, but the vendor was so inflexible about changing its system pricing model that the client killed the project, according to Dan Kaberon, director of computer resource management.

Suppliers are not expected to remain so steadfast for much longer. In addition to grid, technical advances, such as multi-core processors, storage virtualization, on-demand computing and utility computing, are putting pressure on traditional software pricing models. “Increasingly, vendors are becoming much more willing to work with customers to develop pricing metrics that make sense for both parties,” explains Frank E. Gillett, principal analyst at Forrester Research. “One reason is the vendors realize if they don’t work with the customer, a competitor will.”

—Paul Korzeniowski

Major rewrites, minor third-party products
Although many early adopters report success with their grids, building apps on this infrastructure can mean a lot of work. Most apps have to be rewritten to run on a grid. "The trend recently with software development is to make applications more modular, which helps companies that want to move to grid," says Wolfgang Gentzsch, who is working on a grid project for the Microelectronics Center of North Carolina.

Grid can be less complex running internally developed apps, but what happens when a company wishes to run third-party software? The choices are few. New Energy Associates, a Siemens company that provides custom software development and consulting services to energy companies, adapted its software to grid. "Our applications, which include energy market and transmission analysis, energy trading, risk management, and retail and wholesale energy market operation, are complex and processor intensive, so moving them to grid made sense for us," says Neal Tisdale, VPof software development.

Grid standards begin to take shape

A growing number of companies are taking a closer look at grid computing. The ability to manage workloads, monitor usage and charge for resources allocated, and enact comprehensive security capabilities are all important considerations for corporations as they begin to examine deploying a grid application. Because grid computing is still young, most products are based on proprietary technology, and therefore many corporations find it difficult to include all of the desired features in potential grid applications.

To alleviate the problem, users have been pushing suppliers toward supporting industry standards. At the moment, the work is a bit scattershot, but there are signs that vendors may start to coalesce around select standards.

The Global Grid Forum has been working on the Open Grid Services Architecture—a set of specs to make it easier for compute-intensive apps to take advantage of grid computing—and the Open Grid Services Infrastructure—the interfaces and services available to OGSA. The GGF wants to align OGSI with emerging Web services and has been melding OGSI concepts into the Web Services Resource Framework, which is overseen by OASIS.

In the spring of 2004, a vendor consortium that has EMC, Fujitsu, HP, Intel, Siemens, and Sun among its backers, launched the Enterprise Grid Alliance. The organization’s focus is on creating standards for commercial grid apps. The group developed a reference model that features a lexicon of grid terms, a model that classifies the management and lifecycles of the components required for enterprise grids, and a set of use cases that demonstrate the requirements for enterprise grid computing.

The open-source community also has moved toward developing grid products. Companies such as HP, IBM and SAP support open-source software through The Globus Alliance. The alliance’s Globus toolkit helps companies design and maintain grid apps.

The result had been a confusing hodgepodge of specifications and initiatives, but that may change. “Recently, there has been talk about merging the GGF and the EGA, and that step would mean there is a greater likelihood that one set of grid standards would emerge to help users deploy these applications,” notes Frank E. Gillett, principal analyst at Forrester Research.

—Paul Korzeniowski

Grid has not made a lot of sense for other software vendors. "Few off-theshelf applications are grid ready," says Frank Gillett, principal analyst at Forrester Research. Some vendors are leery of changing their software to run on a grid because that architecture may not be as lucrative as traditional systems.

Many corporations avoid building their own apps because it's expensive and time consuming. Companies have turned to internally developed grid systems only when the app promises dramatic business benefits, such as the case with The Hartford Financial Services Group, or when the app is not available from a third party, which is what Entelos experienced.

Not a fine wine yet
Firms that do end up writing grid apps often find such development challenging. Currently, the grid software infrastructure—development tools, management systems—is immature. Part of the problem is the technology's novelty, and another reason is a lack of standard interfaces, a problem that vendors are moving to address.

Grid computing means chopping up large compute problems into manageable bites and distributing them to a variety of systems. Once the processing has been completed, the infrastructure software must be able to put all of the pieces back together—a task that becomes even more challenging when the machines are not running at the same time or working with the same operating systems. Grid software suppliers have been working to add workload management features, which help move information among computers, and failover response functions, which make sure that important transactions are completed.

In addition, extensions are being made to system management tools, such as Computer Associates' Unicenter and IBM's Tivoli, to accommodate grid environments.

Putting procedures in place
Grid also introduces personnel challenges. Most grid applications take processing cycles from servers, but grid proponents have been pushing companies to capture unused desktop and laptop computing cycles. While this could help companies reduce their IT costs, it requires that many put new policies and procedures in place. Many large companies do not have a good grasp on how many computers are in their organization and have to begin with a comprehensive inventory. Once they comple te that task, they need to chart system usage patterns and deduce how much unused processing power may be available to different applications.

Identifying unused processing power is one thing, getting departments to share underutilized resources is another. Corporations tend to operate in stand-alone fashion, with departments responsible for specific applications, but that structure changes in new consolidated grid environments. "Someone has to be in charge of determining when to take cycles from one machine or another," says Enterprise Applications Consulting's Greenbaum. "Some managers may not want to let go and share their resources because they feel they'll lose control over their computers, lose part of their budget or lose control of their workflow."

Entelos experimented with incorporating desktops as part of its grid, but removed them because they added problems instead of horsepower. Bangs explains, "Grid does not work well with laptop users, unless they leave their laptops in the office all of the time, which usually isn't the case. Currently, grid software does not do a good job in recognizing when someone is dialing in from a remote location and figuring out a way to use that system."

While grid technology has potential, its long-term future is unclear. "Grid isn't well suited to applications that require a lot of updating," asserts Hewitt Associates' Kaberon, who thinks the technology will remain a niche rather than a mainstream application development option. For example, in many ERP and CRM systems, one action is dependent upon the result of a previous one, so transactions, such as selling a specific airline seat, can not be broken into several tasks and distributed across a grid.

Others are more optimistic about grid's future. "If the movement in software development is away from singlethreaded to multi-threaded applications and away from hierarchical to component- based application development, then a growing number of applications should become more suited to running on grids," concludes Forrester Research's Gillett. "If those assumptions are correct, the question then becomes: How long will the migration to grid take? Judging from past experiences—a lot of corporate data is still stored in relational database management systems—the software industry is looking at a long-term rather than an immediate change."

ILLUSTRATION BY TODD DAVIDSON

On ADTmag.com