Testing Web services: Even more complex
- By Johanna Ambrosio
Distributed components are certainly not brand new; they have been around for
at least 15 years, if not longer. (The answer as to how much reuse is really
going on is still an open one, however.) But using components as part of a Web
services architecture is a much younger concept, and one that most shops are not
yet employing on a mission-critical basis.
That is just as well. ''There's a lot of stuff missing from Web services
testing,'' said Jason Bloomberg, senior analyst at ZapThink, a consultancy in
Waltham, Mass. ''It's not on any company's radar'' quite yet.
With Web services testing, you would be well-served to take all the advice
given for components in general, and then ratchet it up a few notches in terms
of difficulty and the number of things that can go wrong. The whole idea behind
Web services is embedding several services into one, often on the fly. So one
never knows what combination of services will ultimately result, or where they
will all come from or go to perform their various services. This makes the idea
of testing them that much more difficult. The services that ultimately combine
with your component may not even be in existence at the time you write it.
''It's still not clear how to test orchestrated sites, especially if there's
more than one company involved,'' Bloomberg said.
He sees agile development as ''the only approach to testing Web services in
corporate environments,'' because the test is developed, the component passes
and then a fully tested, scaled application grows gradually. ''Everything you
produce is fully tested; it passes every day,'' he explained.
Gartner Inc.'s Theresa Lanowitz points to CardCops.com as an example of a Web
service that could not pass. Introduced in June 2002 with great fanfare,
CardCops.com checks a database to see whether your credit card has been used
fraudulently. But a few days after its debut, CardCops.com crashed under a
volume of visitors that ''it just could not handle,'' Lanowitz said -- almost a
half-million hits in a little more than two days. Her take: insufficient load
testing of the service, which was unavailable for about a day but is now up
But there is hope on the tools front. Parasoft's SOAPtest uses a WSDL file to
create a ''white-box'' test for Web services. WSDL describes the Web service,
how it will get information from multiple systems, what the call will be and
what the result is supposed to look like. Parasoft's tool looks at the WSDL file
and then tests to make sure the Web service actually performs that function. The
tool can also simulate a client to test the server, and vice versa.
Cape Clear Software Inc., Campbell, Calif., sells a Web services development
environment called CapeStudio that includes testing as part of its suite. (It
also promises to convert Java, COBOL and CORBA systems into Web services.) And
Microsoft is promising a tool to turn an existing COM or DCOM application into a
Web service that plays in its .NET architecture.
As good as that Microsoft tool sounds, though, it could cause some problems,
said Peter Varhol, a product manager at testing vendor Compuware. ''You'll take
a component designed for a single, tightly coupled application and then turn it
into one that doesn't know where the request is coming from or when, or what
will be asked for. That introduces some potential performance implications,'' he
Indeed, load, stress and scalability are key factors to test in Web services,
most observers agree. And Web services need to be tested for compliance and
compatibility to ensure they will be able to play well with others. The Web
Services Interoperability (WSI) Organization - founded by IBM, Microsoft, BEA
and others in February -- will likely play a role here by providing a SOAP test
suite. (You can find out more about the WSI at www.ws-i.org.)
But the majority of this is down the road, where most customers need it to
be. Tyler McDaniel, director of application strategies at the Hurwitz Group in
Framingham, Mass., said that in his firm's survey of 300 enterprises on the use
of Web services technology, ''testing did not come up as part of Web services
deployment right now.'' That is not to say it should not be, he hastened to add
-- just that implementers are not worrying about testing quite yet.
As testing becomes important for Web services, McDaniel said, it will
naturally tie into the business side. If a Web service is seen as critical - as
a revenue generator, as part of another product or for the company's reputation
- then there will be resources thrown at it to ensure its success. Testing will
be part of those resources. ''That's really a business decision,'' he said.
''And this is how IT relates to the business proposition behind a Web service.''
In the meantime, like anything else, the better you understand your Web
service -- who it is meant for, who will use it or who is already using it, peak
load times and other factors in the usage equation -- the better shot you will
have at testing it.
Adam Wallace, vice president of research and development at Cleveland-based
Flashline Inc., said that testing Web services is ''like testing anything else;
there's the testing of the actual service and then the testing of the
application that makes use of that service.'' He acknowledged that ''it
introduces more variables'' to build an application based on multiple Web
services, but noted, ''You can do exception checking on your end of the
application. Theoretically, contractually and technically, the bottom line is
that your application can handle any hiccups in the process.''
See the related stories, '' Testing key to component
quality,'' '.NET and
Java: No real integration yet' 'Representative testing
tools ,' 'Create your
own tests for Java/EJB code' and 'Extending the testing
Johanna Ambrosio is a freelance writer based in Marlborough, Mass., specializing in
technology and business. Contact her at firstname.lastname@example.org.