News

SOA Testing: Reaching the Entire Enterprise

• By Chris Kanaracus
• March 9, 2007

The way Jim Murphy sees it, Windows Communication Foundation (WCF) presents both promise and peril for corporate developers tasked with developing a service-oriented architecture (SOA).

Microsoft's new .NET technology for developing and executing Web services-based distributed systems works as advertised, he says -- but that's the problem.

"[WCF] makes getting going and writing Web services easy to do. The danger that exists is that you'll create a Wild West ... [of] this profusion of Web services because they're easy to do, and you miss the whole point of a [SOA]," says Murphy, vice president of product management at New Hampshire-based Mindreef Inc., a SOA testing solutions company.

This is where companies like Mindreef come in. The company is among a number of firms touting solutions and platforms that can test SOAs on aspects such as interoperability, performance and functionality.

Some vendors in the SOA testing space are focused on just Web services -- key SOA components, but hardly the whole cow. A fully implemented SOA will reach all the way from desktops in the accounting department to the legacy mainframe in the basement.

To this end, companies like Mindreef compare SOAs to an ecosystem, and stress that testing them requires input from and reach into the entire enterprise.

"There's typically a defined group [within an organization] that has the defined role of testing Web services," says Frank Moreno, Mindreef's vice president of marketing. "We look at SOA quality. That's much more a collaborative, multi-role responsibility."

Primed for Growth
Mindreef underwrote a study by Hurwitz & Associates last year that surveyed 99 IT executives from North American and United Kingdom companies with more than 250 employees. Of the companies surveyed, 66 percent had begun deploying a SOA but only 22 percent of the companies doing so had a quality-assurance plan in place.

This situation is set to dramatically change, according to Jason Bloomberg, an analyst with ZapThink LLC. "We definitely see the area of SOA quality as set to explode, and we predict that 2007 will be a breakout year for this segment," he says in an e-mail.

Bloomberg breaks down the SOA testing landscape into three discrete areas. "The first level is Web services testing, with products that test service consumers and providers to ensure proper exchange of messages, and performance testing to ensure scalability and reliability," he says. "This is where many vendors are now, but it's not really SOA testing at all -- it's simply Web services testing.

"The second level is integration/dependency testing, which ensures service-oriented business applications (SOBAs) work properly and enables companies to test the underlying infrastructure. A few vendors have started offering these capabilities," he continues. "The third level is metadata testing, which tests the ability to change contracts, policies and SOBA logic. iTKO [Inc.] tests SOBA logic, but other than that, nobody's at this point yet."

While the SOA testing market is still in its infancy, the SOA concept itself is taking hold in the enterprise.

Christophe Telep, product manager in IBM Corp.'s Rational division, which includes SOA testing tools, says SOA has seen a maturation from small proof-of-concept exercises to implementations that are truly "business-driven."

"They're really starting to have a more global SOA approach. I guess we've all seen that in the market, and we're seeing the same with our customers," he says.

Team-Based Solutions
Mindreef's SOAPScope Server product involves an integrated set of tools that can be used by people without programming skills or deep knowledge of XML. This approach maintains the capability for focused testing by a specific team, but there's also the opportunity for collaboration throughout the business.

SOAPScope Server's collaborative nature allows organizations pursuing a SOA to establish clear standards and policies for the architecture, according to Mindreef.

Other features include the ability to let users test services offline by simulating real-life conditions. That way, tests won't interrupt or overload Web services already up and running in an organization, Moreno says.

iTKO's approach also stresses collaboration among the different groups involved in creating and changing SOA components. The company offers an enterprise-level SOA testing platform, LISA, as well as a series of more focused solutions, including LISA for .NET.

iTKO stresses "functional governance," says iTKO CTO Ruston Vickers in an e-mail. "These services and the full business processes they're a part of must be validated continuously to prevent the 'Wild West' syndrome. This is the whole notion that business agility breeds unintended consequences and the only way to ensure harmony is an automated monitoring of this agile environment," he says.

One portion of iTKO's strategy involves "continuous validation," where regression is checked continually as builds come together. "This allows issues to be caught at every code check-in before [they] ever affect the rest of the team," he says.

Companies moving toward SOA often face challenges in utilizing important but isolated components. Addressing this, LISA 3.5 features "in container" testing, which allows users to stage tests on components that are only accessible directly through the server. LISA also has support for analyzing crucial legacy assets, such as CORBA business logic. Overall, SOA testing won't be effective unless it's done in a way that reflects a true SOA's enterprise-wide reach, Vickers says.

"SOA testing and governance require complete coverage of the technology stack. It's not enough to validate at just single service end-points, and the brutal truth is that SOAs are not just Web services," Vickers says. "You must get people involved in testing these moving parts early and form a full-coverage net around the whole SOA structure."