The SOA-wise enterprise
- By Alan Radding
- June 1, 2006
Gartner expects service-oriented architectures to provide the basis for 80 percent
of new development projects by 2008.
Most enterprises today, however, report having launched only isolated Web services
projects. Despite the fanfare about services, few orgs have a good grasp of how
to deploy these architectures enterprise-wide. What's still needed are clearly
defined best practices for deploying and managing SOAs, which can help orgs identify
worthwhile initiatives, establish governance, button down security and so on.
The Big Idea
Fundamentals of SOA
- Find a legitimate business need by focusing on a messy problem that can be
fixed with a few granular services.
- Establish a framework for governance in conjunction with your SOA architecture
design and fill in the governance details as you go.
- Build in security and identity management from the start; do not rely only
on existing network security.
- Plan from the outset for how you will manage and troubleshoot your SOA
environment as it grows larger.
Some industry gurus claim best practices already exist. Others argue that it is
too early. Too few enterprise-scale SOA developments exist for solid best practices
to emerge, they contend. Early adopters can only tell you what worked for them.
Still, short of stumbling around in the dark, here are some practices around business
value, governance, security, and management. The more involved issues such as
testing, orchestration and reuse come into play but later in the game.
Need for best practices
The State of Utah is one example. Utah's environmental department is using Web
services to share information with the federal Environmental Protection Agency.
The state's criminal justice system relies on Web services to speed access to
data in systems operated by a slew of cities, counties and state agencies. Through
the magic of Web services, "a policeman can pull over a motorist and look
up all kinds of information from different systems," says Randy Hughes, technical
architect, State of Utah, office of the CIO.
At this level, SOA is simple. "Think back to EDI," Hughes suggests.
"You need an architecture for exchanging information. Web services make it
easy." However, Utah's Web services only provide read-only access to the
data. "Today, we cannot run high-volume transactions," Hughes explains.
So what's holding up Utah's effort? "We need to lay out the whole foundation,"
Hughes says. "What kind of data is available, how it is accessed, how it
That's the hard part. The state also needs a comprehensive approach to security
and identity management. In short, Utah, like most orgs jumping into SOA, needs
the kind of guidance only a set of SOA best practices can provide.
Five Points of Entry
IBM has identified five entry points to SOA:
- People—projects that improve productivity by facilitating access to data.
- Process—projects that improve processes starting at the department level, and then rolling out enterprise-wide.
- Information—projects that integrate information through federated views of data.
- Connectivity—projects that speed messaging among heterogeneous systems while ensuring quality of service.
- Reusability—projects that encourage and reward service reuse.
SOA best practices can help orgs at every stage of the SOA process, from identifying
worthwhile SOA initiatives to establishing effective governance. Other initial
process concerns include buttoning down security and managing the whole thing
once it is deployed.
Best practice 1: Identify the business problem
No app-dev project should start without a good business justification, but the
problem seems worse with SOA. "The tools make it so easy. You can take any
legacy component and turn it into a service," says Paul Lipton, senior architect,
enterprise systems management at CA. As a result, "companies just skip the
Not every project lends itself to SOA or makes business sense. "My techies
want to try SOA with the payroll system," says the chief architect at a Houston
aerospace and defense contractor. "OK, the payroll system is old and needs
updating. But do they really want to do what amounts to putting payroll data on
the Net? That's what I call the cool-tool syndrome."
On the other hand, the company has an aging, mainframe-based asset management
system that tracks machinery and equipment. Many internal groups need to access
and update this system. Subcontractors and outside business partners also rely
on it. Faced with the high cost of providing access to all these users, and high
attrition within the ranks of aging mainframe experts, the business case for SOA
here looked very good.
"We did this with SOA because of all the messaging involved," says the
chief architect. "It made sense from a business standpoint. But even then,
this is only a temporary fix. Eventually, we're going to have to transition to
At many companies, the techies decide to start with a neat little project that
can score a fast, easy win. "They are starting in the wrong place because
even when they succeed, they won't have helped the business," asserts Doug
Houseman, director of Capgemini, a consulting firm with a large SOA practice.
Instead, he advises the dev team to "go into the messy areas, the areas everyone
is scared to touch. That way you get some impact from your SOA." Messy areas
typically are those where things change fast.
Once you have identified a messy area, focus on one particularly messy step and
build one or two services that clean it up. For example, a long term customer
may have overdue invoices due to unresolved customer service issues. It would
be good if the accounts receivable system could communicate with the customer
service system before sending a pay-up letter or cutting off the customer. A few
small services could check various systems and kick out the customer for special
handling under certain conditions. "You want granular services here so you
can keep up with the changes," Houseman says. See separate story, "Five
points of entry."
Best practice 2: Address governance at the start
The architecture determines the standards, protocols, tools and technologies the
enterprise will use to build services. Governance addresses issues such as which
services to develop in the first place, how they will be managed, who owns them,
and how they will be secured.
Tools in this space include Infravio X-Registry Platform, Mindreef Coral, and
Systinet Governance Interoperability Framework. In May, 15 vendors-AmberPoint,
Infravio, JBoss and SOA Software, among others-joined forces to support SOA Link,
an end-to-end governance interoperability initiative.
Developers, however, tend to discount governance. They treat it as another administrative
detail they can put off until the end, like filing time sheets. Not so. "Governance
is a big deal. It must be part of the architecture review from the start,"
insists Robert Kelley, partner at systems integrator LiquidHub in Pennsylvania.
Governance entails the specification of the policies under which the SOA will
function. "It addresses the construction of the services and the use of the
services," Kelley explains.
Many IT teams learn the hard way that governance should have been addressed early
on. Railinc, a subsidiary of the Association of American Railroads, encountered
this problem. The company maintains a repository of all the rail equipment of
the Class-1 U.S. railroads. Sharing this information is critical if rail cars
from different railroads are to be connected and moved. Railinc turned to SOA
to give it internal and external access to this data. "By using SOA, we don't
have to dictate what systems our customers can use," says Garry Grandlienard,
director of enterprise architecture.
Using IBM WebSphere, Grandlienard's developers began by identifying services they
needed and started building them. "We are going over our application portfolio
looking for reusable services," he says. One service might check with the
master equipment database. Another might retrieve certain characteristics of a
piece of equipment. A notification service might send e-mails or faxes.
"In each case, we have to think through the interface, figure out what parameters
are required, and what we can make available to the user," Grandlienard says.
Today, Railinc has multiple project teams building services and everyone wants
to reuse services built by other teams. Until this point, the effort has progressed
very well except for one thing. "We have not yet addressed the governance
issue," he admits.
Now Grandlienard is trying to add governance after the fact. "We're working
informally with the development managers. This is a sales job."
Coming up with a comprehensive set of governance principles, however, is a daunting
challenge. One that may put the dev team in the middle of corporate politicking.
"You need governance but you don't need to solve all the problems of the
world," says Jim Crew, formerly the director of data center infrastructure
at Merrill Lynch and now a VP at SOA Software. Instead he recommends going with
something that is good enough for now even if it isn't complete or perfect.
ZapThink Senior Analyst Jason Bloomberg agrees with this approach: "We recommend
you have a governance framework, but you don't have to work out all the details
before you start. Otherwise, you'll end up with paralysis by analysis." Bloomberg
suggests pinning down a few key governance principles to start, such as how services
will be reused and by whom.
When you're a small shop, it's easier to cut corners. With only four of its eight
developers working on SOA, "we're small enough to be self-governing,"
says Sean Carey, software architect at SPS Commerce, Minneapolis. "We just
work out everything among ourselves as needed." SPS offers hosted and managed
B2B services. The company turned to SOA to handle file-based integration and to
give customers easier access to its backend infrastructure.
Enterprises will move through three phases of SOA governance, says Sajay Sethunath,
chief architect in Bearing Point's financial services group. The first phase consists
of policies and standards to govern the underlying XML fabric. The second phase
addresses standards around the repository, management of service level agreements,
and enterprise-wide interdependencies. The third phase governs the crossing of
organizational boundaries and closely resembles Business Process Management.
Best practice 3: Think security
Like governance, security has to be considered from the start. Orgs need to specify
who is responsible for each and use directories to manage the information.
"With the current state of SOA, security is left to you," says Utah's
office of the CIO's technical architect Hughes. "You have to figure out how
to set it up. You really need an identity management system. We currently use
the Utah master directory, but it is not a full-blown ID management system."
Relying on network security is not enough, warn early adopters and analysts. "By
designing security in upfront, you head off some big problems," advises the
defense contractor's chief architect.
ZapThink's Bloomberg, who calls security the first roadblock to SOA, agrees: "People
think that SOA is XML-based so a network firewall is all you need. Well, a firewall
is not enough."
He recommends implementing a wide range of emerging SOA security tools. These
tools, such as AmberPoint SOA Management and SOA Software XML VPN Controller,
act as intermediaries, often in the form of appliances. For example, XML accelerators
and firewalls check the traffic against policies and lists, validate the XML schema,
block malformed XML, and verify authentication and authorization. SOA gateways
serve similar functions. Other appliances can federate identities among multiple
systems. Products here include IBM Data - aPower XS40 XML Security Gateway and
SOA Software's XML VPN Appliance.
"We do not have enough data to identify best security practices," says
Dennis Gaughan, research director of AMR Research, Boston. "A lot of the
security models today are built at the application level. I think we need to get
more fine-grained than that. With SOA, we need to look at security for small pieces
of an application."
Capgemini's Houseman agrees: "The big difference with SOA security is that
you have to do it at the service level, not at the application level."
To accomplish this, the service must be well documented. "You have to say
in plain business language what this service is, what it does, what happens under
which conditions, and the consequences of doing it wrong," he explains.
Best practice 4: Don't forget management
Like security, it's important to address the management issues at the start of
an enterprise-scale SOA project.
The first SOA project is generally easy to manage-just a couple of services, a
few users and light traffic. But what happens when your SOA efforts succeed? Now,
IT is looking at dozens, if not hundreds, of services used by thousands of users.
Services that are reused repeatedly in new composite apps. What happens when a
service fails or performance degrades to the point where users notice?
"Today, much of the management must be done manually," says LiquidHub's
Kelley. "The management tools are primitive."
IT traditionally has managed networks, apps and infrastructure separately. With
SOA those distinctions blur. "You need to integrate the management of all
parts of IT," he adds.
Despite the loosely coupled nature of SOA services, "there are still a lot
of dependencies," CA's Lipton points out. And because services are intended
to look like a black box, he says, "you don't know what those services are
doing inside," which complicates management.
Orgs need "to create a plan to manage those services," Lipton advises.
Specifically, IT teams must figure out how to see into and control the service.
Tools such as Actional Looking Glass (acquired by Sonic Software, which is owned
by Progress Software), AmberPoint SOA Management System, Infravio X-Registry,
IBM WebSphere and Tivoli products, CA Unicenter and others are mature enough to
manage effectively, according to ZapThink's Bloomberg. He recommends a metadata-driven
management infrastructure built around policies, SLAs, and truly independent services.
Still, enterprises early into their SOA effort are not quite ready to invest in
management tools for their fledging projects. "We looked at AmberPoint and
Actional but didn't see any value given where we are now. We just don't have enough
services," Railinc's Grandlienard says.
It's important to remember best practices aren't magic. They only point out what
IT dev teams should be doing and suggest how to do it. If you create poor policies
or fail to use updated threat lists, your governance and security will suffer
despite best practices.
Developers and managers too often regard best practices the way kids regard admonishments
to, say, brush after every meal. Yet, when problems crop up, the highly paid consultants
go first to these best practices and almost invariably find the source of the