Columns

The incredibly unique testing requirements of Web apps

• By Graham Titterington
• May 31, 2001

Rapidly expanding Web applications pose new challenges to software developers. Because they support manufacturing operations in areas such as just-in-time supply chains, and end-user buying and selling, the accuracy and reliability of these apps needs to be high. But current software development processes do not meet all the needs of these new applications, particularly in the area of quality assurance.

Web-based applications continuously evolve. Network configurations change and new types of software are added with little overall control. Web pages are frequently moved, requiring changes to the links of any applications that use them. Users may upgrade their browsers, and other applications may be added to the Web to compete for the same resources as existing applications. Similarly, the hardware and software on Web servers frequently changes. This creates a need for frequent re-testing of Web applications, even when the application itself has not changed.

In addition, the Web's competitive environment depends on speed and reliability. Sites with poorly performance can irritate users, who are only a click away from the competition. Business transaction errors in Web-based applications can result in immediate financial loss, as there is no human in the transaction loop to perform "sanity checks." The speed and volume of Web business can cause large losses to accrue before errors are even noticed. Once again, testing is the only defense against damage resulting from such software errors.

In traditional development environments there is little doubt about the importance of testing critical applications. There are many potential approaches to testing, each of which involves a different degree of automation. Most large projects will use some bespoke tools in the testing process. For Web applications, however, the cost/benefit balance between automated and manual testing shifts greatly toward automated testing due to the amount of repeat testing required. For example, tests need to be performed on all the platforms the application will run on, under varying levels of load, and when the application or its environment changes.

The complexity of Web application testing eliminates the possibility of building bespoke testing tools for a particular project or organization. However, commercially available tools do not yet address all the concerns of Web application owners and users.

What needs testing?
Software testing demonstrates that a piece of software is fit for its intended purpose. This typically includes checking the following:

· The software should deliver the correct results, using enough sets of inputs to give an adequate level of confidence that it will work correctly for all sets of inputs it will meet in use ("functional testing").
· The software should perform adequately, handle any foreseeable workload and deliver its results within an acceptable time while using an acceptable level of resources ("performance testing").
· The software should conform to criteria from either the organization that will use the software or from an external organization - such as a regulatory body, or a customer of the organization using the software.
· The system delivered should be usable in its intended role, considering human/ergonomic and environmental factors.
· The system should be rugged and reliable, as well as handle the failure of its own components.
· The data used by the system should be secure.

In addition to the above, Web applications require particular emphasis on usability, performance and security. Indeed, security is an issue that has not often been included in the tester's brief in the past.

Link testing and load testing are essential for all Web sites, even when they are only providing information and not actually transacting any business. Yet load testing of the type used for traditional architectures is not adequate for the Web - testers must confirm that they are getting sensible responses to service requests and not just "server busy" messages. When an application uses the output from a Web server, it is essential to check that the application responds to any denial of service in a constructive way. This can only be achieved by either including a background load element within the functional testing phase or by specifically simulating all possible component failures. A limited amount of functional testing needs to be included in every load test; hence, the distinction between load testing and functional testing is not clearly defined in the Web application domain.

Technically, Web pages are complex entities. Several mark-up languages are used to define the content of Web pages. A single page can contain numerous different types of objects: text, links to other pages, interactive dialogs, images, audio or video clips, Java applets, ActiveX components and so on. These exemplify three very different behaviors:
· Fixed objects that do not change while they are displayed.
· Interactive objects that send new output to the user interface in discrete blocks in response to user input or messages from the Web server.
· Dynamic objects that continuously change their output without requiring any external stimulus.

Testing dynamic content requires a new approach. Conventional testing, whether automated or manual, is based on the comparison of specific outputs. It is not useful to regard a dynamic output (visual, audio or otherwise) as a sequence of millions of static outputs even if this is a correct physical interpretation. A more practical solution is to split the testing into a two-phase process: Validate that a particular combination of programs, scripts and data can generate an acceptable output, and test that the Web application delivers this combination of outputs to the client terminal.

Performance testing is central to Web application testing. However, the performance of these systems is dependent on the network -merely stress testing a Web server or an application server will not in and of itself render useful information. Preliminary work needs to be done to identify key factors in the performance of the specific application so that they are simulated during the test. Some testing tools can now simulate the use of a range of IP addresses in a test session, a useful step toward making load tests more realistic but one that is still a very short of a simulation of the network.

The appearance of a Web page is determined by the browser on which it is displayed, and modified by user options set on the particular browser. Because the behavior of a browser can be modified using features such as cookies, content must be tested at a logical level rather than at the level of comparing screen displays. It may also be useful to test the operation of an application on a range of browsers.

Over the last two years, testing tool suppliers have made considerable progress adapting GUI-based tools for use on Web browsers, simulating load on a variety of servers, and recognizing the interrelationships between load testing and functional testing. The explosion of e-commerce offers them the best opportunity they have had. There remain a number of user needs that are not satisfied by the current offerings. This will keep vendors busy for some time to come.