In-Depth

Wake up and smell the Java

• By Lana Gates
• January 1, 2001

The world of software development is much different today than it was just a few years ago. Developers are under so much pressure to produce applications quickly that they do not have much time to test them. Yet if the apps are not tested and fail, it is a public embarrassment, because the majority of software applications today are designed for Web browsers and the like. Because of this, developers can face bigger losses than they did back in the client/server days when time was more plentiful.

Welcome to the development paradox. What can a developer do to ensure a quality product that will not fail and leave online customers frustrated? The answer, of course, is to perform thorough testing. Testing is of grave importance, quite literally, when it comes to e-commerce applications. If e-commerce apps fail, essentially so does the company responsible for the site. "The competition is just a click away," as the saying goes.

Java is the language of choice for numerous e-commerce applications for a number of reasons. Developers are increasingly using Java in the different application tiers, not just in the GUI. Java applications make up the backbone structure of many e-commerce applications. It is a highly structured language with a wide variety of vendors producing their own packages for it. It is more open to a variety of development approaches than, say, Visual Basic, which is provided only by Microsoft, or C++. Java is more portable because it interacts with Web browsers and not directly with operating systems. Enterprise JavaBeans (EJBs) make developing Web applications much easier.

Because of this portability, a system using Java is more complex and opens itself up to more points of potential failure. A wide variety of Java packages means a greater demand for Java applications to be tested thoroughly. Because Java is relatively new, and is less mature than C++ or Visual Basic, it has, in many respects, fewer standards. "There's not a proven record of performance here," said Steve Kansa, product line sales support manager at Compuware Corp., Farmington Hills, Mich.

In addition, again because of Java's portability, the different possibilities in testing Java applications pose a challenge. "Java is covering a larger and larger scope of different applications all the time," Kansa noted. "There [is] a wide variety of different fields to cover."

The trick in testing Java applications is to test iteratively throughout the entire development life cycle to ensure quality in every piece of the application. Traditionally, client/server developers left testing until the very end, which might seem to indicate that testing was not a high priority for them, but that was not the case. Of course, they wanted to develop quality products. But that type of development did not allow for testing earlier in the process; it required a complete product.

Testing Java applications, on the other hand, does allow for earlier testing because Java is used to build component-based applications. The components can be tested by themselves before the product is complete.

Developers may not be amenable to testing. Why should they be? They believe they have created a quality product. It is hard for them to be objective about it. They are oriented toward developing, not testing, which is the reason testing has traditionally been the responsibility of the quality assurance department.

That stereotype is changing, however, just as application development has changed during the past few years. Developers are realizing that earlier testing eases development later in the process. In fact, Rational Software Corp., Cupertino, Calif., developed Rational QualityArchitect (RQA) at the request of developers who wanted to test earlier in the development process.

RQA lets developers test components before the system is completely built. It tests EJBs and allows developers to interact directly with them to do automated unit testing. The product helps developers model EJBs with Rational Rose or reverse engineer them into Rose, then generates tests from that model. RQA allows developers to test one specific function on one specific JavaBean or to test many specific functions on many specific JavaBeans.

Rational also offers Java support in its Rational Suite TestStudio—where testing comes at the end of the process—and its Purify, Quantify and PureCoverage products. Choice Hotels International, headquartered in Silver Spring, Md., uses Rational Suite PerformanceStudio, Rational Suite TestStudio, TeamTest and ClearQuest in its software testing endeavors.

The hotel chain develops applications for internal use, as well as for its public Web site. The quality assurance department supports 16 to 17 applications and is responsible for following those applications through their entire life cycles, including regression testing on each new build, requirements tracking and technical requirements.

"We try and get in [testing] as early as we possibly can in the development life cycle," explained Chad Mason, manager of test automation at Choice Hotels in Phoenix, Ariz. "Jumping in at the end and testing right before production slowed us down and pushed back release dates."

As part of early testing, the test automation staff created a suite of automated scripts. "To run through a full test of one of our applications used to take us 10 days to two weeks; we can now do it in less than eight hours," Mason declared. His department previously tested manually, which led to inconsistencies. The biggest challenge for the group was coming up with a repeatable methodology. "Now all of the testing we do is repeatable," Mason said. "We always have a thorough test. There are a lot fewer defects going out in production at this point."

Rational is not the only software vendor offering Java testing support. Compuware supports functional Java testing in its TestPartner and QARun products. QARun makes it possible to test front-end applications built in Java. TestPartner is specifically geared toward Web applications running through a browser. It supports Java and Microsoft technologies through Visual Basic for Applications. Compuware supports performance and Java load testing in QALoad, which can simulate and scale-up users to meet certain service-level agreements.

Mercury Interactive Corp., Sunnyvale, Calif., is yet another supporter of Java testing. The company provides load testing through its LoadRunner product, and functional testing through its WinRunner and XRunner products. LoadRunner tests for scalability on the server, which is usually built and designed before the front end for browser-based products. Once the server is tested and the front end is installed, the entire system is ready for load testing.

Functional testing is the next phase, using XRunner for Unix front ends and WinRunner for Windows-based front ends. In addition, Mercury is releasing new support for Java in Astra QuickTest, a functional testing product specifically designed for Web-based applications.

RadView Software Inc., Burlington, Mass., also supports Java in its WebLoad and WebLoad Resource Manager products designed specifically for testing Web-based applications. Segue Software, Lexington, Mass., includes Java support in its testing products, as well. Segue's SilkPilot provides functional testing for Java servers, including EJBs. SilkTest tests Java in a browser. SilkPerformer supports load testing of Java applications.

Too good to be true?
Parasoft Corp., Monrovia, Calif., entered the Java testing scene about five or six years ago with a product that takes a completely different approach to testing. Jtest was designed to test and prevent errors in Java applications, rather than to simply take errors out of applications once they are written. The product reads source code to determine how to test, and then automatically creates test cases based on that information.

This sounds impossible. If you have 100 lines of code, for example, you could have a decision statement every five lines, which would greatly increase the possibility of making it harder to find bugs. "The complexity of the program is overwhelming," stated Adam Kolawa, CEO at Parasoft, "and people believe it's impossible to do. People thought gene therapy was impossible, too," he added.

Parasoft calls this technique error prevention. Jtest helps developers write code in such a way that the probability of making mistakes is much lower, Kolawa explained. "Testing by itself is really ineffective. That's why it costs so much and is so slow. You can do more testing, but the return on that is very small," he said. The key, said Kolawa, is to change the way you write code and how you prevent errors in the first place.

Kolawa likened software testing to testing automobiles in the 1970s. The major car manufacturers at the time were trying to put crews of workers at the end of the production line to test every car before shipping it to the dealer. Toyota, he said, started to look at every error and correlate it to the appropriate station on the production line, reducing the probability of making an error at each station. "If you can reduce the probability of making mistakes on the production line, you'll reduce thousands of errors in production," Kolawa explained.

"The sooner quality gets in the code, the better it is because the longer you wait to catch strange behavior, the harder it is to diagnose it," agreed Alberto Squassabia, software engineer at Hewlett-Packard Co., Fort Collins, Colo. "When you wrap that into one to five layers, the symptoms may become misleading," he added.

Squassabia believes the biggest advantage of Parasoft's Jtest is that it takes all the groundwork out of exercising any testing at a fine granularity level. "What Jtest does is a very tedious job of functionality verification of all parameters," he said. Its only downfall, he added, is that it does not automatically do algorithmic verification. "It doesn't know if you're working on sorting an algorithm or socket server, so it cannot exercise the application with knowledge of the domain in which it will be operated," explained Squassabia.

Quality, like anything else, comes at a cost. Squassabia believes Jtest's penetration in the market would be greater if Parasoft lowered its price. The last licenses he purchased cost between $3,000 and $3,500 each, and his company has five floating licenses.

Along the same lines as Jtest, Rational offers TestFactory, which tests Java applications without any user involvement. A user provides the name of an executable Java application, and TestFactory starts clicking buttons and choosing menu options in an attempt to open every window of the application. It typically runs over a period of four to six hours and puts various data into fields to find out where an application crashes. Running over a length of time not only tests more thoroughly, but also ensures that the application will work over longer periods without crashing.

Although that sounds pretty intelligent, the product is sometimes called a "dumb monkey tester," because it does not really know what data to put into which fields. It might enter a phone number and a credit card number when asked for a user ID and password, for example. It works using random data and strings of varying length to provide reliability testing. Reliability testing is one of three tiers—the others are unit and functional testing.

Testing blurs the lines
As the development life cycle continues to shrink, it produces a greater need for testing throughout the entire life cycle. As a result, the line is blurring between QA and research and development, which is not necessarily a bad thing. Actually, it is producing better quality products with fewer bugs. Testing is being accepted as a core part of development, providing a more collaborative effort, especially where Java-based Web applications are concerned. "We're not seeing walls being built in the Java area, whereas maybe with client/server walls were being built," noted Jonathan Rende, Mercury's vice president of testing.

With Java being used in the GUI as well as in the middle and back tiers of e-commerce applications, testing has had to change. Now developers and QA departments test Java components in the servers first, then work back out to the end-user perspective, finishing with the front end. Test automation makes this a much easier process.

"As with testing any application, the business criticality of the software is the driving factor in how much you test and the way in which you put [the application] through rigorous testing practices," noted Melinda-Carol Ballou, senior research analyst at Meta Group Inc., Westborough, Mass. E-commerce applications have tremendous business criticality. Therefore, it is imperative that they be tested thoroughly. "They must be tested or companies face serious risks and prohibitive costs, such as revenue and reputation," said Ballou.

What it boils down to is that testing and quality go hand in hand, a pairing that will never change.