In-Depth

The SOA-wise enterprise

• By Alan Radding
• June 1, 2006

Gartner expects service-oriented architectures to provide the basis for 80 percent of new development projects by 2008.

Most enterprises today, however, report having launched only isolated Web services projects. Despite the fanfare about services, few orgs have a good grasp of how to deploy these architectures enterprise-wide. What's still needed are clearly defined best practices for deploying and managing SOAs, which can help orgs identify worthwhile initiatives, establish governance, button down security and so on.

Some industry gurus claim best practices already exist. Others argue that it is too early. Too few enterprise-scale SOA developments exist for solid best practices to emerge, they contend. Early adopters can only tell you what worked for them.

Still, short of stumbling around in the dark, here are some practices around business value, governance, security, and management. The more involved issues such as testing, orchestration and reuse come into play but later in the game.

Need for best practices
The State of Utah is one example. Utah's environmental department is using Web services to share information with the federal Environmental Protection Agency. The state's criminal justice system relies on Web services to speed access to data in systems operated by a slew of cities, counties and state agencies. Through the magic of Web services, "a policeman can pull over a motorist and look up all kinds of information from different systems," says Randy Hughes, technical architect, State of Utah, office of the CIO.

At this level, SOA is simple. "Think back to EDI," Hughes suggests. "You need an architecture for exchanging information. Web services make it easy." However, Utah's Web services only provide read-only access to the data. "Today, we cannot run high-volume transactions," Hughes explains.

So what's holding up Utah's effort? "We need to lay out the whole foundation," Hughes says. "What kind of data is available, how it is accessed, how it is published."

That's the hard part. The state also needs a comprehensive approach to security and identity management. In short, Utah, like most orgs jumping into SOA, needs the kind of guidance only a set of SOA best practices can provide.

SOA best practices can help orgs at every stage of the SOA process, from identifying worthwhile SOA initiatives to establishing effective governance. Other initial process concerns include buttoning down security and managing the whole thing once it is deployed.

Best practice 1: Identify the business problem
No app-dev project should start without a good business justification, but the problem seems worse with SOA. "The tools make it so easy. You can take any legacy component and turn it into a service," says Paul Lipton, senior architect, enterprise systems management at CA. As a result, "companies just skip the value proposition."

Not every project lends itself to SOA or makes business sense. "My techies want to try SOA with the payroll system," says the chief architect at a Houston aerospace and defense contractor. "OK, the payroll system is old and needs updating. But do they really want to do what amounts to putting payroll data on the Net? That's what I call the cool-tool syndrome."

On the other hand, the company has an aging, mainframe-based asset management system that tracks machinery and equipment. Many internal groups need to access and update this system. Subcontractors and outside business partners also rely on it. Faced with the high cost of providing access to all these users, and high attrition within the ranks of aging mainframe experts, the business case for SOA here looked very good.

"We did this with SOA because of all the messaging involved," says the chief architect. "It made sense from a business standpoint. But even then, this is only a temporary fix. Eventually, we're going to have to transition to something else."

At many companies, the techies decide to start with a neat little project that can score a fast, easy win. "They are starting in the wrong place because even when they succeed, they won't have helped the business," asserts Doug Houseman, director of Capgemini, a consulting firm with a large SOA practice.

Instead, he advises the dev team to "go into the messy areas, the areas everyone is scared to touch. That way you get some impact from your SOA." Messy areas typically are those where things change fast.

Once you have identified a messy area, focus on one particularly messy step and build one or two services that clean it up. For example, a long term customer may have overdue invoices due to unresolved customer service issues. It would be good if the accounts receivable system could communicate with the customer service system before sending a pay-up letter or cutting off the customer. A few small services could check various systems and kick out the customer for special handling under certain conditions. "You want granular services here so you can keep up with the changes," Houseman says. See separate story, "Five points of entry."

Best practice 2: Address governance at the start
The architecture determines the standards, protocols, tools and technologies the enterprise will use to build services. Governance addresses issues such as which services to develop in the first place, how they will be managed, who owns them, and how they will be secured.

Tools in this space include Infravio X-Registry Platform, Mindreef Coral, and Systinet Governance Interoperability Framework. In May, 15 vendors-AmberPoint, Infravio, JBoss and SOA Software, among others-joined forces to support SOA Link, an end-to-end governance interoperability initiative.

Developers, however, tend to discount governance. They treat it as another administrative detail they can put off until the end, like filing time sheets. Not so. "Governance is a big deal. It must be part of the architecture review from the start," insists Robert Kelley, partner at systems integrator LiquidHub in Pennsylvania.

Governance entails the specification of the policies under which the SOA will function. "It addresses the construction of the services and the use of the services," Kelley explains.

Many IT teams learn the hard way that governance should have been addressed early on. Railinc, a subsidiary of the Association of American Railroads, encountered this problem. The company maintains a repository of all the rail equipment of the Class-1 U.S. railroads. Sharing this information is critical if rail cars from different railroads are to be connected and moved. Railinc turned to SOA to give it internal and external access to this data. "By using SOA, we don't have to dictate what systems our customers can use," says Garry Grandlienard, director of enterprise architecture.

Using IBM WebSphere, Grandlienard's developers began by identifying services they needed and started building them. "We are going over our application portfolio looking for reusable services," he says. One service might check with the master equipment database. Another might retrieve certain characteristics of a piece of equipment. A notification service might send e-mails or faxes.

"In each case, we have to think through the interface, figure out what parameters are required, and what we can make available to the user," Grandlienard says.

Today, Railinc has multiple project teams building services and everyone wants to reuse services built by other teams. Until this point, the effort has progressed very well except for one thing. "We have not yet addressed the governance issue," he admits.

Now Grandlienard is trying to add governance after the fact. "We're working informally with the development managers. This is a sales job."

Coming up with a comprehensive set of governance principles, however, is a daunting challenge. One that may put the dev team in the middle of corporate politicking. "You need governance but you don't need to solve all the problems of the world," says Jim Crew, formerly the director of data center infrastructure at Merrill Lynch and now a VP at SOA Software. Instead he recommends going with something that is good enough for now even if it isn't complete or perfect.

ZapThink Senior Analyst Jason Bloomberg agrees with this approach: "We recommend you have a governance framework, but you don't have to work out all the details before you start. Otherwise, you'll end up with paralysis by analysis." Bloomberg suggests pinning down a few key governance principles to start, such as how services will be reused and by whom.

When you're a small shop, it's easier to cut corners. With only four of its eight developers working on SOA, "we're small enough to be self-governing," says Sean Carey, software architect at SPS Commerce, Minneapolis. "We just work out everything among ourselves as needed." SPS offers hosted and managed B2B services. The company turned to SOA to handle file-based integration and to give customers easier access to its backend infrastructure.

Enterprises will move through three phases of SOA governance, says Sajay Sethunath, chief architect in Bearing Point's financial services group. The first phase consists of policies and standards to govern the underlying XML fabric. The second phase addresses standards around the repository, management of service level agreements, and enterprise-wide interdependencies. The third phase governs the crossing of organizational boundaries and closely resembles Business Process Management.

Best practice 3: Think security
Like governance, security has to be considered from the start. Orgs need to specify who is responsible for each and use directories to manage the information.

"With the current state of SOA, security is left to you," says Utah's office of the CIO's technical architect Hughes. "You have to figure out how to set it up. You really need an identity management system. We currently use the Utah master directory, but it is not a full-blown ID management system."

Relying on network security is not enough, warn early adopters and analysts. "By designing security in upfront, you head off some big problems," advises the defense contractor's chief architect.

ZapThink's Bloomberg, who calls security the first roadblock to SOA, agrees: "People think that SOA is XML-based so a network firewall is all you need. Well, a firewall is not enough."

He recommends implementing a wide range of emerging SOA security tools. These tools, such as AmberPoint SOA Management and SOA Software XML VPN Controller, act as intermediaries, often in the form of appliances. For example, XML accelerators and firewalls check the traffic against policies and lists, validate the XML schema, block malformed XML, and verify authentication and authorization. SOA gateways serve similar functions. Other appliances can federate identities among multiple systems. Products here include IBM Data - aPower XS40 XML Security Gateway and SOA Software's XML VPN Appliance.

"We do not have enough data to identify best security practices," says Dennis Gaughan, research director of AMR Research, Boston. "A lot of the security models today are built at the application level. I think we need to get more fine-grained than that. With SOA, we need to look at security for small pieces of an application."

Capgemini's Houseman agrees: "The big difference with SOA security is that you have to do it at the service level, not at the application level."

To accomplish this, the service must be well documented. "You have to say in plain business language what this service is, what it does, what happens under which conditions, and the consequences of doing it wrong," he explains.

Best practice 4: Don't forget management
Like security, it's important to address the management issues at the start of an enterprise-scale SOA project.

The first SOA project is generally easy to manage-just a couple of services, a few users and light traffic. But what happens when your SOA efforts succeed? Now, IT is looking at dozens, if not hundreds, of services used by thousands of users. Services that are reused repeatedly in new composite apps. What happens when a service fails or performance degrades to the point where users notice?

"Today, much of the management must be done manually," says LiquidHub's Kelley. "The management tools are primitive."

IT traditionally has managed networks, apps and infrastructure separately. With SOA those distinctions blur. "You need to integrate the management of all parts of IT," he adds.

Despite the loosely coupled nature of SOA services, "there are still a lot of dependencies," CA's Lipton points out. And because services are intended to look like a black box, he says, "you don't know what those services are doing inside," which complicates management.

Orgs need "to create a plan to manage those services," Lipton advises. Specifically, IT teams must figure out how to see into and control the service.

Tools such as Actional Looking Glass (acquired by Sonic Software, which is owned by Progress Software), AmberPoint SOA Management System, Infravio X-Registry, IBM WebSphere and Tivoli products, CA Unicenter and others are mature enough to manage effectively, according to ZapThink's Bloomberg. He recommends a metadata-driven management infrastructure built around policies, SLAs, and truly independent services.

Still, enterprises early into their SOA effort are not quite ready to invest in management tools for their fledging projects. "We looked at AmberPoint and Actional but didn't see any value given where we are now. We just don't have enough services," Railinc's Grandlienard says.

It's important to remember best practices aren't magic. They only point out what IT dev teams should be doing and suggest how to do it. If you create poor policies or fail to use updated threat lists, your governance and security will suffer despite best practices.

Developers and managers too often regard best practices the way kids regard admonishments to, say, brush after every meal. Yet, when problems crop up, the highly paid consultants go first to these best practices and almost invariably find the source of the problem.