News

Meeting the challenges of integration testing

• By Dan Romanchik
• July 18, 2003

Matthew Van Vleet is co-founder and CTO at Solstice Software, which recently released Version 3.6 of Integra Enterprise, a suite of integration testing tools. I had the opportunity to ask him some questions about the challenges of integration testing.

Q. What is the difference between the testing done during an integration project and the testing done during an application development project?

A. There are several key differences. First, integration project architectures are message-based, not code-based, and the failure points are generally in the messages, not in the code as they would be in an application development project. Unfortunately, most testing tools are focused on testing code and GUIs, not testing messages. As a result, developers must create code stubs or elaborate error-prone traces during end-to-end tests.

Testing in an integration project needs to focus on the messaging. Both the business and technical team members need to think about message flows and logic, and the tools they use should be designed to support messages. Currently, most people use GUI tools to "drive" messages, and then spend hours and days tracing defects. Message testing tools, on the other hand, are designed to test at the message level and to pinpoint message problems.

Another key difference is that integration projects may involve many different applications. This creates both organizational and technology challenges. Organizationally, teams need to think not just about their code, but how it affects both input and output components. Most application development teams don't have the vision or the technology to validate their systems operation in the overall enterprise. An integration project may also include many different technologies, such as XML, MQ, middleware brokers, TCP/IP and others. Integration teams need tools that work with many technologies so they can validate the technology handoffs, as well as the business logic.

Finally -- and most importantly -- it is almost impossible in a large integration project to orchestrate an end-to-end test with application test tools. What integration projects need is the ability to test early in development. They need to unit test, and component test, pieces of the logic before all the pieces are available.

Q. What are some of the most common mistakes that you've seen customers make in an integration project?

A. Most of our customers don't realize how difficult it is to do integration testing. Most integration solutions promise that they will streamline development and architecture -- and they do -- but the complexity, paths and permutations still remain and need to be tested. You still have to test all of the complex business logic, which is now under [or in] another layer of integration-ware.

Not only is the testing more complex, but it is also a different kind of project to manage. As I mentioned earlier, the testing is message-based, there may be many different protocols to test and there is the logistics challenge of having all the components ready. Unless you consider all these factors, it's easy to underestimate the work involved by as much as a factor of four.

Q. How can these mistakes be avoided?

A. We informally surveyed a group of project managers and found that to avoid integration testing problems, you should build a different kind of team, focus on interconnections right from the requirements phase, plan and manage an assembly phase, and use tools focused on integration testing.

Q. What kinds of things should developers watch out for in the future?

A. We think that integration projects are going to force application developers to come out of their cocoons, and to understand and work with the larger enterprise context. For some, this will be good; for [others,] this will be a nightmare.

If you look at the integration methodologies that are starting to come to the surface, such as Gartner's Integration Competency Center and Accenture's Integration Model, they all make developers more responsible for quality. It used to be that a developer was responsible for only his or her own code. In the future, however, they will be responsible for not only the code, but how it integrates with other applications.

This will force application developers and development groups to add more "integration knowledge" and to acquire tools that support this added responsibility. The team must now focus on the big picture and how applications interconnect, not just on how the application works in isolation.

Links:

For other Programmers Report articles, please go to www.adtmag.com/article.asp?id=6265