In-Depth

Is now the time to manage Web services?

• By Johanna Ambrosio
• July 1, 2004

Web services management is one of those strange areas. When it is needed, it is critical; and when it is not needed, it is almost certainly moot.

That is the state things will remain in for at least the next two years, most independent industry watchers agree, until more customers devote themselves to building an IT architecture with Web services at the core.

In the meantime, most customers simply do not require that level of infrastructure, said Tom Welsh, a senior consultant in the enterprise architecture practice at Cutter Consortium, Arlington, Mass. He used the following analogy to explain his stance. “If you’re in a rural environment, perhaps you have a few people driving around, and when you get to an intersection you keep your eyes open to avoid running into other drivers,” he noted. But when the small town grows to a point where there are many more people driving around, the government will likely need to install traffic lights, stop signs and other regulatory items.

While some IT organizations are moving forward with Web services management, most customers are still wandering through the Web services backwoods. That is not to say that people are not developing Web services, because they are. In fact, just about everyone this side of mom-and-pop shops has at least a few Web services to call their very own and is likely using even more services that have been developed by outside groups. Frank Kenney, a research analyst at the Stamford, Conn.-based Gartner research group, said Web services are “like potato chips; once you have one, you can’t stop. They have such enormous power for doing things like integration, and lapping up legacy logic to expose to different sets of customers.”

But building and consuming a few one-off SOAP-based services is a very different matter than mindfully creating an entire infrastructure devoted to Web services and their ilk. It is this second group of shops -- a fairly small number right now, by most accounts -- that require a heavy-duty management component.

The need for management also depends on how a company is implementing Web services, analysts explain. If a customer using full-fledged SOAP has more than, say, 20 or 30 Web services, they will eventually hit critical mass and require a way to track it all. But if a developer chooses to implement a Web service in a lighter-weight protocol -- XML-RPC, for example -- then chances are the need for a management solution may not be as great.

“It’s the difference between building a garden shack and a 20-story office building,” Cutter’s Welsh explained. “If you’re building the latter, you have to get things right from the start, and management and security are necessary evils. But if they’re not necessary, you’re much better [off] not having them.”

When the much-anticipated focus on Service-Oriented Architectures (SOAs) does occur, it is anyone’s guess as to what becomes of the providers. The majority of today’s bevy of small, dedicated management software providers may well be overtaken by IBM, Microsoft and other large platform providers. These large vendors will provide management features as part of their larger Web services product set, even if they do not do so now.

In fact, Uttam Narsu, principal analyst in the application development and infrastructure research group at Forrester Research in Cambridge, Mass., said he is “somewhat skeptical” that Web services management will remain a standalone market in the long run. “The fundamental value proposition of a SOA is about ease of management. So the big-name infrastructure providers will be supplying Web services management as part of their offerings,” he noted. That will take at least two years to shake out, Narsu said.

Even when it does, Narsu and other analysts agree, there will be room for some of the strongest and best-run of the independents to survive. For customers of whichever vendors are acquired or fold, the standards-based nature of Web services will help make the transition to other products somewhat easier.

It is a good thing that this shakeout is expected to take a while, as some of the truly important standards are just now wending their way into products. WS-Security, for instance, was ratified by the Organization for the Advancement of Structured Information Standards (OASIS) as a full-fledged standard in April 2004. And the key management standard -- also being shepherded by OASIS -- is the Web Services Distributed Management (WSDM, pronounced “wisdom”) Model, which will allow you to manage different pieces of a Web services infrastructure from different vendors, and to agree on things like alert structures and the like. This standard is still in its first version and, as of this writing, there has been no consensus on even that tentative beginning.

A small, fractured market
In the meantime, vendors in the Web services management arena have been playing a near-constant game of musical chairs. “There are a lot of vendors, and they’re all trying to figure out their niche,” explained Jason Bloomberg, senior analyst at consultancy ZapThink LLC in Waltham, Mass. “They’re all changing their story and moving around,” trying to figure out the most appropriate mix of features to meet customers’ needs.

Another factor keeping some would-be customers away is pricing, which generally ranges between $150,000 and $250,000 for a complete management suite. “I’d expect that competition will drop this [price] down slightly,” said Forrester’s Narsu, “but price points will remain high for the time being. The major price competition will come from the infrastructure vendors over time.”

The major players in the management market include Actional, AmberPoint, Blue Titan, Digital Evolution, HP/Talking Blocks, Flamenco Networks, Infravio, Itellix and Westbridge Technology. Most provide software, but some -- including Flamenco -- offer services to help customers outsource the Web services management task completely.

With Web services management flying from so many different directions and players, there is one under-represented constituency: vendors of development tools. To make maximum effective use of Web services, “applications will have to be encoded to do things like execute if/then statements based on a network’s conditions,” said Gartner’s Kenney. “I think the end-game takes place in the development world,” but we are a long way away from there, he added.

There have been some tentative steps to this type of environment, though. In the past few months, both AmberPoint Inc., Oakland, Calif., and Actional Corp., Mountain View, Calif., have introduced free-of-charge developer versions of their respective management products. These are admittedly not the full-blown management packages, but the notion is to help developers begin to build-in manageability from the beginning of the Web services process instead of retrofitting these features at the end.

Both vendors have their developer packages available for download off their respective Web sites. Actional’s version is called SOAPstation.

AmberPoint Express will be bundled with Microsoft’s Visual Studio 2005 and will allow developers to have fault detection and “some proof point that Web services are working and working well,” said Ed Horst, AmberPoint’s vice president of marketing.

Another big difference from even a year ago, noted ZapThink’s Bloomberg, is that “last year we saw soup-to-nuts functionality,” where every vendor implemented virtually every feature one could ever want in a management package. Now, however, some suppliers are starting to specialize in certain niches -- managing or creating service-level agreements, for instance, or audit and version control. Some are targeting security as a core feature, while others are working to ensure that their management suites work with existing Web security leaders like VeriSign and Netegrity.

Different from systems/network management
To anyone who has been around the IT industry for a while, the notion of when and how to start proactively monitoring and managing Web services is very likely familiar territory. After all, entire sub-industries have sprung up to do just that for systems and networks, dating back to the 1970s if not before. Candle, IBM, Computer Associates and others made huge amounts of money in these areas back in the day, and their successors for the client/server computing model include names like Tivoli.

Granted, it is not exactly the same thing. Managing Web services is more complicated, most agree. “There’s an opportunity to view a problem quite differently,” said AmberPoint’s Horst. “At the systems level, the light is red or it’s green. At the Web services level, you can see who the Web service is interacting with, how the order is flowing through, and if individual transactions are being handled properly. You can see inside the actual XML payload.”

He cited a situation with an airline a few years back, whose Web site advertised a round-trip ticket from Los Angeles to Paris for $25, and sharp-eyed consumers flocked to the deal. “The system was live for the better part of two hours,” he said. “This cost [the airline] $150,000 in lost revenue. But because transactions were processing and credit cards were cleared, nothing caught the error” before it was too late. In Web services monitoring, presumably that would not happen. “We can interrogate the transaction and see that all is well,” claimed Horst.

Dan Foody, CTO at Actional, said that the Web services realm is more about managing relationships and less about dealing with the underlying systems. “In a traditional model, most people built projects as standalone entities. Every project was pretty much an island.” Take a bond-yield calculation -- in a large brokerage house, there might be 19 of these floating around, each powering a separate application. If any of these went down, the other 18 are not usually affected.

But now “we’re moving to having one bond-yield calculation engine that’s used by 19 different applications,” Foody said. “Now that I have that, it’s really bad if that bond-yield engine goes down. And the reality is that I have 19 different users, each with different performance profiles.” Every customer cannot be treated the same.

Overall, he said, “there’s a measure of truth in saying that Web services management is part of something bigger. But what is that bigger thing? Any SOA is a combination of loosely coupled and tightly coupled apps. Web services is not a great technology for tight coupling today, so you’ll need a mix -- you might want to look in legacy databases in ways that are not done with Web services.”

Foody said he wonders how this will play out in customer companies: Will Web services management be brought in as a separate discipline or will it be handled by those who have traditionally managed systems and networks? His guess is that the customers who will make Web services management into a separate specialty will be the ones that have a centralized policy-management group.

“One thing’s for sure: it won’t be defined by the vendors,” he noted.

One customer’s journey
Thomson Prometric is an online testing firm based in Baltimore. It made its name in 1990, back when it was called Drake, when it developed a computerized test for Novell to administer its groundbreaking Certified Network Engineer certification. Now Thomson Prometric provides on-screen tests for driver’s licenses and other environments in addition to IT.

The company has grown, in part, through a series of acquisitions that helped to create a stovepipe infrastructure that is so typical in many organizations. That, in turn, “hindered our ability to innovate and create service offerings because of our inability to integrate,” said Chris Crowhurst, vice president and principal architect at the firm. “We wanted to remove these separate channels, and I was looking at the best integration platform,” including EAI and other technologies, he explained.

Luckily, however, the company had managed to build up a layer of XML abstraction within its systems. “In every one of our systems, XML was created as a tier to help communicate, so XML was already present,” Crowhurst explained. So as they looked at that level, the company decided it would be helpful to move to standard protocols, including SOAP. “It was very natural to stay with XML rather than a proprietary integration platform,” Crowhurst noted.

Thomson Prometric has built an integration platform based on Actional management tools that do the transformation of business requests into Web services APIs. (Before selecting Actional, the firm did extensive proof-of-concept tests with five other vendors, Crowhurst said.) Thomson Prometric also uses Microsoft’s BizTalk to create federated queries that go across multiple back-end systems.

Along the way, “we have a view of the entire transaction; we see it performing each orchestration,” Crowhurst said. In addition, Actional’s management suite -- called Looking Glass Server -- allows Thomson Prometric to “get a holistic view of our network services,” he added.

When considering the various tools, it was important for whatever the firm selected to be able to work with HP’s OpenView, Thomson Prometric’s system-management standard. The idea is to allow the data center staff to be alerted to a problem within Actional, and to be able to click through from OpenView to Looking Glass to get information about any problems.

Thus far, the integration Web services count is around 50 end-points. “If we get this right, and the architecture is robust and scales, we’ll see that number ballooning,” Crowhurst said.

But it is not just about the technology. Equally important to selecting the right product suite was establishing a governance model -- a way of managing the processes that constitute the SOA. “We couldn’t just arbitrarily allow people to connect in,” he explained. Then they created policies: that all Web services must be managed, that they all have to use certain authentication and encryption standards, they must meet a certain namespace standard and so on.

Another aspect was being able to “reach out to other organizations that are doing similar work,” said Crowhurst. “There’s a tight-knit community doing true SOA, and everyone is helping everyone else.”

Finally, Crowhurst suggests that customers consider management tools from the perspective of everyone within the organization that will work with them. “If you look at it as an architect, you’ll buy a different tool than you would as a developer,” he explained. But, he noted, it is a tool that will be used by many different layers in the organization -- development, testing, deployment, operations and so on. “We had a set of requirements from each constituency and, before we purchased Actional, we reviewed it with everyone to make sure that they had no objections.”

At the end of the day, it is still very early going for Web services management. “To be able to manage Web services no matter what platform they ultimately execute on will require more standardization to handle things like how services change over time,” said Forrester’s Narsu. “A lot of pieces need to come together.”

Please see the following related stories:

  “A tale of two surveys” by Johanna Ambrosio

“Where management and security collide” by Johanna Ambrosio