Columns

SOAPtest bubbles up

• By Jason Halla
• May 1, 2004

SOAPtest Version 2.5
Parasoft Corp.
Monrovia, Calif.
626-256-3680
www.parasoft.com
Rating: 4 out of 5

Testing Web service apps can be tricky. Generally speaking, a development team in a business-to-business environment is responsible for one end point in a Web services architecture. To verify that your portion of the service is functional, you need the other end point, which leaves you with a choice between waiting on the completion of the other end point or building your own in-house version of the opposite end point to your Web service architecture. Parasoft's SOAPtest provides an automated environment for emulating a SOAP server or client.

SOAPtest is an Automated Error Prevention product for Web services and enterprise apps built with SOAP. Developed and marketed by Parasoft Corp., SOAPtest allows users to verify all aspects of a Web service, from WSDL validation, to unit and functional testing of the client and server, to performance testing. Version 2.5 adds several new features, such as support for the WS-Security spec, MIME attachments (text and binary), and a WS-I Analyzer Tool.

For this review, I installed SOAPtest on a Windows XP box. The product installed without a hitch, though the registration process was a bit onerous. Parasoft requires you to contact its office by phone to register the software and secure a password. Implementing a secure Web-based registration system accessible either from a browser or within SOAPtest (using SOAP, perhaps?) would have been more convenient.

The SOAPtest client is deceptively simple, with 10 large, friendly buttons along the top menu and an address bar for browser-like access to your WSDL file. I say deceptively simple because SOAPtest offers up a huge amount of functionality nested in contextual menus accessed by right-clicking within various points of the interface.

The product takes a Swiss Army knife approach to Web services testing. It provides the ability to emulate a SOAP server or client; perform load and regression testing; and script the app in Java, JavaScript or Python. It also comes with a host of related features, including an integrated Apache Tomcat/Axis server, WSDL/XML verification, XSL/XSLT transformation, XML cleanup, a file Diff tool and an XML dictionary.

SOAPtest's main testing functionality lies in the SOAP client and server emulation feature and the server load testing feature. Both server and client emulation test cases can be manually designed by the user or automatically generated by SOAPtest. Real-life testing scenarios can be generated with relative ease by opening up a WSDL document and clicking a button.

Server functional testing (emulating the client) begins with the designation of a WSDL for SOAPtest to work with. WSDL files can be accessed locally or remotely over HTTP, allowing you to work with WSDL documents that have already been deployed on your server.

All test cases are created and saved as projects within SOAPtest. Each project reviews one WSDL file; you have the option of creating separate tests against the validity of the WSDL file itself when establishing a new project. Once your project has been created, SOAPtest instantly creates test suites for each method described in your selected WSDL file.

Clicking the "Run Tests" button in the menu begins the WSDL functional and (if selected) validation testing. Once the testing is complete, immediate feedback as to the success or failure of each test is presented in the right-hand menu. A list appears in the main window with a brief description of the tests that were performed, which object methods they were performed on, as well as a summary showing whether the test was a success or failure.

You can test the SOAP server further by adding additional virtual users. Each virtual user test can trigger its own set of functional tests, allowing you to create real-world traffic conditions and test your WSDL's functionality under the same kind of strain it will experience when launched as a live service. Regression controls can be added to test cases, allowing you to track changes over time.

Simulated HTTP traffic is recorded for each WSDL method, allowing the user to review the exact SOAP information as it would be returned by the server. Returned SOAP/XML data can be further manipulated through the Diff comparison tool and XSL/XSLT transformation, among other features.

The range of testing possibilities adds up to an extremely robust, comprehensive tool for emulating client interaction with a SOAP server.

While the client emulation testing features are specifically tailored for validating functionality of your WSDL, SOAPtest's load testing capabilities are designed to push your SOAP server to its physical limits. The process can yield invaluable insights about potential breakpoints and security issues your app could encounter.

As with client emulation, each virtual user in load testing can be created with its own usage scenario, set of data sources and so on. A single-copy SOAPtest license permits installation across multiple PCs specifically for the purpose of using unused CPU cycles to generate increased load tests so that additional machines can be added to a load test case. SOAPtest outputs a series of reports on the load test that can be exported in a variety of formats. Version 2.5 introduces customizable HTML reports.

To facilitate server emulation and SOAP client testing, SOAPtest features an integrated Apache Tomcat server. By selecting Tomcat to facilitate server emulation from within SOAPtest, Parasoft helps to ensure that your SOAP client's interaction with the emulated server mimics real-world performance.

Server emulation is especially useful for catching HTTP header issues and problems with the SOAP envelope, as well as the response sent to the client and the client's ability to deal with these issues. Clients can be gauged against a number of Web services concurrently.

All of these features and functions add up to an extremely effective tool for testing SOAP-based Web services. My only complaint -- if complaint is even the right word -- is the price. At nearly $4,000, the product is beyond the means of most independent developers, and possibly many SMBs. A version of SOAPtest with some features scaled back, targeted at these types of customers, would be welcome.

Nevertheless, my hat is off to Parasoft's team for putting together the right combination of capabilities for Web services testing. With its ability to automatically generate relevant testing scenarios, SOAPtest is the perfect counterpart for developing enterprise-class Web service apps. If your Web services are mission-critical, I definitely recommend investigating SOAPtest.

Bottom Line: With its ability to automatically generate relevant testing scenarios, SOAPtest is the perfect counterpart for development of enterprise-class Web service applications. If your Web service is mission-critical, I definitely recommend investigating SOAPtest.

Platforms and Pricing: SOAPtest is available for Windows 2000/XP, Linux and Solaris. Apple OSX is currently not supported.

Pricing starts at $3,995.

Pros:

• Comprehensive suite of testing environments: server, client and load testing.
• Wide range of related tools: XML/SOAP validation, scripting capabilities, real-time reporting and more.
• Automatically generates test cases with just a WSDL file.
• Java-based app, so a variety of platforms are supported.

Cons:

• Steep learning curve: detailed features take some time to master.
• The price is a little on the high side, which could be an issue for independent developers and consultants.