In-Depth

Testing Web services: Even more complex

• By Johanna Ambrosio
• September 30, 2002

Distributed components are certainly not brand new; they have been around for at least 15 years, if not longer. (The answer as to how much reuse is really going on is still an open one, however.) But using components as part of a Web services architecture is a much younger concept, and one that most shops are not yet employing on a mission-critical basis.

That is just as well. ''There's a lot of stuff missing from Web services testing,'' said Jason Bloomberg, senior analyst at ZapThink, a consultancy in Waltham, Mass. ''It's not on any company's radar'' quite yet.

With Web services testing, you would be well-served to take all the advice given for components in general, and then ratchet it up a few notches in terms of difficulty and the number of things that can go wrong. The whole idea behind Web services is embedding several services into one, often on the fly. So one never knows what combination of services will ultimately result, or where they will all come from or go to perform their various services. This makes the idea of testing them that much more difficult. The services that ultimately combine with your component may not even be in existence at the time you write it.

''It's still not clear how to test orchestrated sites, especially if there's more than one company involved,'' Bloomberg said.

He sees agile development as ''the only approach to testing Web services in corporate environments,'' because the test is developed, the component passes and then a fully tested, scaled application grows gradually. ''Everything you produce is fully tested; it passes every day,'' he explained.

Gartner Inc.'s Theresa Lanowitz points to CardCops.com as an example of a Web service that could not pass. Introduced in June 2002 with great fanfare, CardCops.com checks a database to see whether your credit card has been used fraudulently. But a few days after its debut, CardCops.com crashed under a volume of visitors that ''it just could not handle,'' Lanowitz said -- almost a half-million hits in a little more than two days. Her take: insufficient load testing of the service, which was unavailable for about a day but is now up again.

But there is hope on the tools front. Parasoft's SOAPtest uses a WSDL file to create a ''white-box'' test for Web services. WSDL describes the Web service, how it will get information from multiple systems, what the call will be and what the result is supposed to look like. Parasoft's tool looks at the WSDL file and then tests to make sure the Web service actually performs that function. The tool can also simulate a client to test the server, and vice versa.

Cape Clear Software Inc., Campbell, Calif., sells a Web services development environment called CapeStudio that includes testing as part of its suite. (It also promises to convert Java, COBOL and CORBA systems into Web services.) And Microsoft is promising a tool to turn an existing COM or DCOM application into a Web service that plays in its .NET architecture.

As good as that Microsoft tool sounds, though, it could cause some problems, said Peter Varhol, a product manager at testing vendor Compuware. ''You'll take a component designed for a single, tightly coupled application and then turn it into one that doesn't know where the request is coming from or when, or what will be asked for. That introduces some potential performance implications,'' he said.

Indeed, load, stress and scalability are key factors to test in Web services, most observers agree. And Web services need to be tested for compliance and compatibility to ensure they will be able to play well with others. The Web Services Interoperability (WSI) Organization - founded by IBM, Microsoft, BEA and others in February -- will likely play a role here by providing a SOAP test suite. (You can find out more about the WSI at www.ws-i.org.)

But the majority of this is down the road, where most customers need it to be. Tyler McDaniel, director of application strategies at the Hurwitz Group in Framingham, Mass., said that in his firm's survey of 300 enterprises on the use of Web services technology, ''testing did not come up as part of Web services deployment right now.'' That is not to say it should not be, he hastened to add -- just that implementers are not worrying about testing quite yet.

As testing becomes important for Web services, McDaniel said, it will naturally tie into the business side. If a Web service is seen as critical - as a revenue generator, as part of another product or for the company's reputation - then there will be resources thrown at it to ensure its success. Testing will be part of those resources. ''That's really a business decision,'' he said. ''And this is how IT relates to the business proposition behind a Web service.''

In the meantime, like anything else, the better you understand your Web service -- who it is meant for, who will use it or who is already using it, peak load times and other factors in the usage equation -- the better shot you will have at testing it.

Adam Wallace, vice president of research and development at Cleveland-based Flashline Inc., said that testing Web services is ''like testing anything else; there's the testing of the actual service and then the testing of the application that makes use of that service.'' He acknowledged that ''it introduces more variables'' to build an application based on multiple Web services, but noted, ''You can do exception checking on your end of the application. Theoretically, contractually and technically, the bottom line is that your application can handle any hiccups in the process.''

See the related stories, '' Testing key to component quality,'' '.NET and Java: No real integration yet' 'Representative testing tools ,' 'Create your own tests for Java/EJB code' and 'Extending the testing process'