In-Depth

EJBs force testing changes

Forward-compatibility and componentization have finally brought the long-ignored testing tenet "test early, test often"closer to reality. Expanded testing realm is providing "an honorable career" for testers.

Java's strengths can create problems for application development. While it allows developers to build componentized applications, its enablement of distributed computing, as well as its flexibility, make testing difficult. These issues have led to changes in methods for software testing. Increasingly, developers are testing code as they write it, instead of handing over finished code to software quality assurance teams.

Meanwhile, testers are adding white-box testing to the standard black-box and system-level tests, and vendors are expanding their tools' range either by adding more capabilities and tools or by teaming up with partners.

"Test early and test often" is an aphorism well known among developers. But because of the emphasis on rapid application development (RAD) during the heyday of client/server, testing was relegated to software quality assurance teams who looked at code after it was written.

That is not enough when you are moving at Internet speed and software is becoming more important, according to Oliver E. Cole, president of OC Systems Inc., Fairfax, Va. Now, testing is increasingly being conducted at the development stage and it is no longer something to be ashamed of.

"Ten years ago, if you were a tester, you were a failed developer; now testing is an honorable career," Cole said.

Java's ability to integrate with other applications means Java-based applications are forward-compatible with other apps that may not have been built yet. So testing and development staff have to develop and test applications "even before there's a front end to the interface," said Jonathan Rende, vice president of product management at Mercury Interactive Inc., Sunnyvale, Calif.

Michael Rozlog, chief architect and evangelist at Borland Software Corp., Scotts Valley, Calif., said today's integrated software market means developers "have to assume applications developed today will be touched by unknowns." According to Rozlog, Borland has "the largest IDE available for Java development anywhere" in the form of JBuilder.

Vendors are beginning to offer solutions to the problem of forward-compatibility, said Billie Shea, director of Research and Analyst Services at Newport Group, Barnstable, Mass.

One such vendor is Mercury Interactive. Its LoadRunner tool allows users to conduct automatic performance tests of Enterprise JavaBeans even before they have a user interface to an application. When LoadRunner generates unit tests, it also puts begin and end transaction commands around the EJB request so that when the tests are scaled up to multiple requests, performance tests are created automatically.

Testers can use LoadRunner to conduct unit tests, load/performance tests and end-to-end load testing of EJBs. For consistency, testers can reuse the unit test scripts for load testing and run those load tests against the same EJBs on which they had conducted the unit tests.

LoadRunner has a reporting/playback capacity that can automatically capture all EJB-specific information along with methods, constants and variables that are part of each class. It can then either test all the EJBs on a server simultaneously or individually, said Mercury Interactive's Rende. Once users record a transaction or test, they can scale it up to "maybe hundreds of thousands of users," he claimed.

Mercury Interactive also offers functional testing tools—WinRunner and QuickTest. Testers can use these tools to conduct functional testing end-to-end with the user interface at the user level.

The firm's tools offer a combination of white- and black-box testing. The unit and load/performance tests are white-box tests, and the end-to-end load and performance tests are black-box tests, Rende said.

Several tool vendors, including WebGain Inc., Santa Clara, Calif., and Sitraka Inc., Toronto, are partnering with Mercury Interactive to complement their products. Sitraka's tools monitor memory-level components of EJBs, which Mercury's Rende said is very important for unit and load testing of EJBs, as memory leaks are "one of the biggest problems with EJBs."

Sitraka offers the JProbe family of performance-tuning tools to test applications running in a Java virtual machine. JProbe server-side edition has four main components, said Jeff Zado, product line director for JProbe. The JProbe Integrated Memory Debugger looks for memory leaks in the overall memory usage of Java applications. JProbe Profiler looks at the performance characteristics of Java applications. JProbe Threadalyzer performs thread analysis, while JProbe Coverage performs coverage measurements so users can determine how much of the code has been used, based on their use cases.

Third-party testing

Some corporations outsource testing to third-party software testing services like Flashline.com Inc., Cleveland, which offers quality assurance testing of Java components through its Flashline QA Laboratory. Gary Baney, Flashline.com's chief technology officer, said the laboratory recommends and provides three types of testing for components: language standard compliance and design quality, performance and scalability.

All languages have coding standards, or conventions, that have to be followed during the application development cycle, Baney said. Some are globally adopted by the development community, while others are specific to a particular corporation. Flashline.com provides a test engine that tests for globally adopted coding standards, then lets clients implement their own parameters to test for internal standards.

To test for design quality, Flashline.com uses TogetherSoft's Control Center 5.0 product. "Coding standards are the syntax of the source code; design quality is the architecture of the component—its depth of inheritance, its span, the number of links it has to the external object," Baney said. He added that Flashline.com is the only software testing laboratory willing to test components for cyclamatic complexity—how frequently one class or method invokes other methods within the same component. "The more thoroughly the developer has implemented nesting, the higher the level of cyclamatic complexity," he said.

For performance tests, Flashline.com uses Sitraka's JProbe family of tools. It writes a Java wrapper around the tools, extracts test data from them and formats it for developers so they can use it. "If I run Sitraka tests, I get this mountain of data, of which the typical developer needs about 5%," Baney said. "We want to tell developers the 5% of the code on which they need to spend 95% of their time."

Flashline.com uses Empirix's Bean-test to look at EJB response times for scalability testing. Bean-test generates the threads that simulate the external user community, then forces transactions through the beans and maps the number of transactions per second and exception levels.

Flashline.com provides clients detailed analyses and reports on the results of its tests.

—Richard Adhikari

Gary Baney, chief technology officer at third-party testing laboratory services provider Flashline.com Inc., Cleveland, speaks highly of JProbe.

"Finding out what's called most frequently and what consumes most resources—Sitraka does this better than anyone else," he said.

WebGain offers Café, which Mercury Interactive's Rende described as "probably the number one Java development environment." Mercury Interactive and WebGain are integrating LoadRunner and Café so that Café users can generate EJBs and then generate automatic scripts using LoadRunner to test the EJBs.

Automated testing
Mercury Interactive's tools are well known for automating testing. Another player in this area is Empirix Inc., Waltham, Mass.

Empirix offers Bean-test, a functional and load testing tool. Bean-test automates test generation, including the creation of nested complex objects. This is done through a patent-pending algorithm that recursively drills into an object to build up the nested complex object, said Walter Vahey, director of product marketing at Empirix's Web applications test group.

The tester enters a primitive, such as a phone number, into a table and Bean-test drills down into the primitives and populates the nested complex objects. If a tester takes a table, such as a list of phone numbers, and applies it to the load test, Bean-test will iterate through that list and create a different virtual user for each phone number.

Bean-test also allows users to pull records out of their databases and drive "enormous" test loads with their own data, explained Vahey. Alexander Pelaez, director of IT architecture at 1-800-Flowers, Westbury, N.Y., is using Bean-test to test data retrieval of entity beans from different database hosts to determine which host provides optimal performance. He runs Windows NT and Solaris back ends. He is also using Bean-test to conduct performance testing on EJBs developed offshore "to make sure they meet our contracted performance levels," and to test application servers from different vendors to see which gives him the best performance.

The current version of Bean-test, Version 3.1, is optimized for BEA WebLogic, Bluestone Total-e-Server and IBM WebSphere servers, and any EJB 1.1 application server. Empirix supports Linux, Solaris, HP-UX, AIX, Windows NT and Windows 2000, said Empirix's Vahey.

Writing tests yourself

Off-the-shelf testing tools just will not do if you are creating a very specialized or highly complex application.

Kovair Inc., San Jose, Calif., has developed its own XML-based tool to test VIPCenter, its relationship management software package. Built on an EJB server, VIPCenter lets users manage an account relationship across multiple locations, platforms and business processes.

Kumar Goswami, Kovair's CEO, said commercial testing tools are not adequate because "we have fairly complex data structures, and when you have lists of lists, test tools don't do so well." Also, determining the accuracy of test results is not easy and has to be done manually. So, it was easier for Kovair to write its own tests.

When adequate testing tools become commercially available, Goswami will consider using them. For the time being, however, Kovair will stick with its homegrown tool.

—Richard Adhikari

Dealing with componentization
The componentized aspect of EJB application development makes testing difficult because "when you put a bunch of components together into a system, you cannot test all the ways they will be used—there are too many permutations and combinations," said Bill Thornburg, director of marketing at Rational Software Corp., Cupertino, Calif. Also, he added, few developers follow proper procedure, which calls for them to write tests as they create components.

Several vendors offer component-level testing tools for EJB applications. Rational Quality Architect (RQA) from Rational Software automates the writing of test drivers and stubs as components are developed. Part of Rational's Quality By Design products, RQA is based on the concept of extreme programming, where developers write tests before they write the code. If developers are using visual modeling, RQA inspects the model and designs the drivers and stubs for the component that has just been written; this allows developers to test individual components in isolation before assembling them into their system.

This way, "individual developers turn in components of a much higher quality to the integration team," Thornburg explained. With EJBs, RQA uses the introspection technique, where testers ask a component to tell them about itself. RQA then uses the responses to write a series of software tests.

Rational also offers a set of white-box tools in its Quality by Design category. The white-box tools consist of Rational Purify, Rational Quantify and Rational Pure Coverage. Purify finds memory leaks; Quantify profiles the performance of code modules; and Pure Coverage tells developers how much of the code has been executed. It is often used together with functional testing tools.

In addition, Rational offers functional testing and load and performance testing in its Rational Suite TestStudio.

Testing production systems

One of the real bugbears of testing is that it cannot be conducted on production systems. But many problems in production systems cannot be duplicated. Some vendors are now offering tools to test production systems.

InCert Software Corp., Cambridge, Mass., offers TraceBack, which speeds up diagnosis and recovery from problems in both laboratory and runtime environments. TraceBack is enabled directly from Sun Microsystems' Forté for Java IDE.

TraceBack takes a snapshot of the entire environment when a bug manifests. That information, called a trace file, is stored in a repository and can be perused later. It contains the exact sequence of the application's execution, line by line, leading up to the fault. Users see this history correlated with their source code, and can step backwards or forwards through multiple threads to see what happened when the bug manifested.

TraceBack injects its functionality into the application being tested, instead of embedding logic at the kernel level. In a Windows or other conventional application, TraceBack injects inline machine code; in Java applications, it injects Java byte code into the middle of the application "so there doesn't need to be any external source of control or communication," said Richard Schooler, InCert's chief technology officer.

OC Systems Inc., Fairfax, Va., offers Aprobe. This product runs on the AIX, Windows and Solaris platforms, as well as on Java. A Linux version will be launched soon.

Aprobe patches the JVM in memory so, as the byte codes are written in the JVM, it inserts extra byte codes at every trace point, according to Oliver E. Cole, OC Systems' president. The amount of overhead required depends on how much is traced.

Cole said Aprobe does not require class, EJB and JAR (Java Archive) files to be modified, whereas InCert's TraceBack does. Both TraceBack and Aprobe trace method entry and exit points, but Aprobe also traces for the arrival of messages and things occurring, "because that's really where you get the understanding of the problem," said Cole. Aprobe also lets users insert arbitrary Java byte code so users can conduct white-box tests.

—Richard Adhikari

Segue Software Inc., Lexington, Mass., offers SilkPilot, a product that focuses on functional and regression component-level testing. It is designed to test "any Java programs with any public interfaces," said Mark Pinone, the firm's sales engineering manager. SilkPilot can be used to test Java, RMI, EJB, CORBA and IIOP applications.

SilkPilot lets testers build a full transaction visually. So, if a user is building middleware that will deal with a banking application, they can also build a bank account and "full functional transactions that make sense," Pinone said. SilkPilot also provides the user interface to build the transaction. Testers do not need to write the stubs—the Java code that makes calls to and from a system.

While SilkPilot offers testers the black-box approach, it can be used with Segue's other tools—such as SilkPerformer, for load testing and performance simulation; and SilkTest, for GUI-level functional and regression testing—to create system-level tests. Segue also offers SilkTest International, which extends SilkTest's functionality by letting testers perform single-script, simultaneous tests of localized applications across multiple languages, including Mandarin, Kanji and Arabic; multiple platforms; and multiple Web browsers.

Class Integrated Quality Inc. (ClassIQ), Wilmington, Del., offers IQTest for Java, a component testing tool used at the enterprise level or to test Java classes. IQTest has a simulation feature that lets users test an application throughout the entire life cycle instead of waiting until specific development landmarks are achieved. This eliminates project dependencies, said Joe Burns, ClassIQ's senior vice president.

IQTest for Java acts as a plug-in to relational databases, so users can save both their tests and the test results to any relational database, Burns said. IQTest for Java provides a standard way of looking at attributes and all values, which allows users to store data so they can conduct destructive testing. It also provides full reporting through every step of the testing, which provides a fine granularity of supervision, he said. For example, development managers can pinpoint a particular class and ask why there are no test cases for it, or pinpoint the developer who wrote a particular section of code, according to Burns.

IQTest for Java offers a front end so developers do not have to write their own code to conduct tests. This omission saves a lot of time because, Burns said, "developers spend about 50% of their time testing what they develop."

Another feature of IQTest for Java is ClassIQ's FlexAdapter technology. This is an API that lets users "integrate all the enterprise features, such as simulation features, group dependencies, sharing test cases, saving test cases, automatically replaying test cases and full reporting, into our main product for any new protocol that has not previously been encountered, without having to reinvent the wheel," said Burns. For example, FlexAdapter let MCI Worldcom integrate tests for a protocol called Registry, which ClassIQ had not encountered previously, into IQTest within three weeks. "Their estimates showed it would take them six to 12 months to build a testing piece around Registry otherwise," Burns said.

ClassIQ has integrated IQTest with Mercury Interactive's LoadRunner and TestDirector tools. This will allow ClassIQ to widen its horizons because "IQTest focuses on early testing and Mercury focuses on deployment once testing is done," Burns said.

Most   Popular