News

A dialog on testing Extreme Programming

• By Dan Romanchik
• May 16, 2003

Most of us realize that adopting Extreme Programming (XP) techniques has a big impact on application development. But what about testing? To answer this question, Lisa Crispin and Tip House have written "Testing Extreme Programming" (Addison Wesley Professional, 2003). Programmers Report asked them several questions about the topic.

Q: How is testing different in an XP project?

Lisa: The biggest difference is that on an XP project, the entire development team takes responsibility for quality and all the testing tasks, including automating the acceptance tests.

Tip: Testing in a traditional project is often a one-time event that occurs at the end of the project, whereas testing in XP starts before code is written and goes on continuously. This makes testing in an XP project a common, central activity for the entire team, rather than a peripheral one involving just a few members or an outside group.

Q: How is the role of the tester different in an XP project?

Lisa: As Ron Jeffries says, XP is not about 'roles,' it's about a tight integration of skills and behaviors. In XP, testing is an integrated activity. The team needs continuous feedback with the customer expressing needs in terms of tests, and programmers expressing design and code in terms of tests. On an XP team, the tester will play both the customer and programmer 'roles,' focus on acceptance testing, and provide leadership in the areas of testing and quality assurance to the rest of the team.

Tester activities on an XP project include negotiating quality with the customer, clarifying stories, enabling accurate estimates, making sure acceptance tests verify that the quality specified by the customer is met, helping the team to automate tests, helping the team to produce testable code, and forming an integral part of the continuous feedback loop that keeps the team on track.

Tip: Because of the intense integration of all activities on an XP team, being a tester on an XP team is a lot less 'lonely' than in a traditional software project. Instead of being an outsider, possibly even an antagonist, a tester on an XP team is as much an insider as anyone else, and may even write a significant chunk of the code. But because of this, the expectations of a tester on an XP team are apt to be much higher, both in terms of technical expertise and in the level of professionalism they bring to the table.

Q: Are the tests run in an XP project different than those run in a project using a more traditional development method?

Lisa: The biggest difference between XP projects and most 'traditional' projects is the concept of test-first design. With XP, every chunk of code is covered by unit tests that must pass all the time. This allows acceptance tests to be just that -- Does the code do what the customer wanted? The acceptance tests define the level of quality that the customer wants and pays for.

Tip: Generally, yes. There are many, many more unit-level tests run on an XP project and a lot fewer high-level tests. The high-level tests on an XP project tend to focus almost exclusively on the question 'Does the system satisfy customers requirements?' rather than on 'Does the code do what we intend?'

Q: For testers new to XP, what's the one thing they should keep in mind?

Lisa: Testers who are new to XP should keep in mind the XP values: communication, simplicity, feedback and courage. Courage may be the most important. As a tester, the idea of writing, automating and executing tests in speedy two- or three-week iterations, without the benefit of traditional requirements documents, can be daunting.

Testers need courage to let the customers make mistakes and learn from them. They need courage to determine the minimum testing that will prove the successful completion of a story. They need courage to ask their teammates to pair for test automation. They need courage to remind the team that we are all responsible for quality and testing. To bolster this courage, testers on XP teams should remind themselves that an XP tester is never alone -- your team is always there to help you.

New XP testers should also remember that it's up to the customer to define the quality they want. We testers tend to hold software to our own personal standards. If, for example, the customer doesn't want to spend enough resources to make a Web application handle 1,000 concurrent users, and I break it with 1,000 concurrent users, that isn't a bug. Since the acceptance tests are the customer's way to specify the level of quality [they] want, the XP tester plays a key role.

Tip: [Testers new to XP should keep in mind that] there is no abstract standard of quality or 'goodness' to which you must hold the project software. That standard is dynamic and must be defined, discovered and developed with the customer and the other members of the team as the project progresses.

For links to information on "Testing Extreme Programming," including sample chapters, go to http://www.amazon.com/exec/obidos/tg/detail/-/0321113551/qid=1053112633/sr=8-1/ref=sr_8_1/104-2681745-0099954?v=glance&s=books&n=507846

Links: