In-Depth

Is Corporate America ready for XP?

• By Johanna Ambrosio
• August 1, 2001

Now more than five years old as a defined programming methodology, Extreme Programming (XP) is catching on, albeit very slowly. At this point, it has more of a toehold than a foothold in corporate IT. That position is poised to change, adherents say, as Corporate America discovers XP's many benefits. Proponents say, if it is given a fair, XP can help companies develop working code that is remarkably bug-free in a very short amount of time. The resulting code is also easier to change later than software that has been created using the traditional waterfall approach, XPers add.

"XP to date has been an innovator's thing, where technical people are doing it for technical reasons," said Kent Beck, one of the three original XP inventors and director of Three Rivers Institute in Merlin, Ore. "But that won't be forever." Beck is currently working to get his creation accepted into more "mainstream" companies. (See the sidebar "Kent Beck's elevator pitch.")

Matt Light, a research director at Gartner Inc., Stamford, Conn., said that XP is "spreading slowly. It's certainly being used more than it was three years ago."

Ron Jeffries, another of the three original XP guiding lights and now a consultant with Object Mentor Inc., Vernon Hills, Ill., figured there are several hundred ongoing XP projects these days, up from about 100 a year ago.

There are also now books and conferences, training classes and user groups, consultants and XP coaches—an entire infrastructure that did not exist earlier. The main XP conference, held in Italy in May, went from 150 attendees to 250 attendees in the span of two years, Jeffries added, and the first major XP conference in the U.S. was held in Raleigh, N.C. in July.

Beck compares the acceptance rate of XP with that of any major new way of doing business. "Just-in-time manufacturing took 10 or 15 years to catch on, and lots of people still don't get it," he said.

"What's extreme about it is the way it tries to meld speed and quality," Gartner's Light explained. Speed is addressed by having small, frequent releases with as simple a design as possible, and then by constantly upgrading the code (known as "refactoring" in XP-speak). Quality is enforced by running tests before the coding is done and by embedding inspections and peer review through pair programming. (For a more in-depth introduction to XP and its 12 core practices, see "'Extreme' method simplifies development puzzle," by John K. Waters, ADT, July 2000, p. 20, which is also available at www.adtmag.com.)

Doing it differently
But while adopting "XP proper" is not rocket science, by design, it is also not a no-brainer. "I would venture to say that those who expect it to be extremely lightweight are finding it's not," said Light.

XP certainly does not require, and in fact outright disparages, the type of up-front planning and documentation that the heavyweight methodologies define. On the other hand, there is much involved with doing XP, and even more to doing it well. "It's not lighter than air, and it's not lighter than RAD [Rapid Application Development]," Light said.

It is also not something that other members of the IT organization are bound to take lightly, said Joshua Kerievsky, founder of Industrial Logic Inc., an XP and software-pattern consultancy in Berkeley, Calif. XPers often clash with the database group and the QA department, among others in a traditional IT setup, he said.

"In the database world, they're used to a lot of detailed analysis and design and up-front thought before they do anything. As far as the database people are concerned, the XP people have a couple of screws loose," Kerievsky explained. According to him, the mistrust is mutual because the XP people consider database folks to be "old and stodgy, forcing XPers to go more slowly" than they want to go.

There is also the issue of the way XP changes the way developers work with the ultimate customers of the software. In many IT shops, developers do not deal directly with the software's end users; that is the purview of business analysts and managers. But XP demands that developers, managers and customers all sit down together to agree on what must be done and when, and then they are all accountable to one another. So XP changes the entire feedback cycle of what is being delivered and when it is delivered.

For these reasons, Kerievsky and others suggest the need for XP change agents within and outside IT. These agents should be people who can bring the highest levels of the company together to agree that XP is the methodology of choice—whether it is company-wide or for one particular project.

Convincing naysayers
Besides culture clashes, part of the reason that adopting XP requires an uphill battle belongs to Corporate America's well-chronicled reputation for being resistant to change. And part of the problem belongs to XP itself. Although XP claims to simplify the software development process and yield better code and happier customers, even rabid XPers lack many hard numbers to prove their promises.

Jeanine de Guzman was a member of the Ford Vehicle Cost Analysis Profit System (VCAPS) team, which was one of the early XP implementers; she is now an enterprise information architect with the Dearborn, Mich.-based company. "It's hard to put into terms the business value behind XP, and that's why it's not getting rampant acceptance," she said. "Half of what XP promises sounds like buzzwords. Unless you have cost numbers, it's hard to be sold."

One of the few who does have statistics is Laurie Williams, an assistant professor at North Carolina State University in Raleigh, N.C. She has done two studies related to XP. The first, when she was teaching at the University of Utah several years ago, tested the concept of pair programming with 42 seniors at the school. Of the 42, some 14 worked alone and the rest worked in pairs.

During the course of their different programming assignments, the number of hours decreased for the pairs. So, by the third assignment, the pairs spent only 10% more time than did the singles. For example, if an individual spent 10 hours on the assignment, the pair worked together for five hours and 15 minutes. At the same time, the pairs passed 15% more of the post-development test cases—and more than 90% said they enjoyed programming more.

More recently, Williams did another test—this time at a major computer manufacturer she declines to identify publicly. The experiment involved one programming task that the company estimated would take 90 hours using the traditional waterfall method of development. With pair programming, it took 60 hours—and "the quality was much better," she said. "There were only two defects. They did not tell me what their norm was, only that this was a lot better than that." Williams is currently working with Vienna, Austria-based Allianz Insurance on more XP research that should yield results by the end of the summer.

Current practitioners report that XP seems to work best in small teams of fewer than 12 programmers, on projects where it is used as the only methodology and it is used from the inception of the project. It is also most effective when it is teamed up with object-oriented technologies. This is not to say that XP cannot be used with SQL databases or with larger groups of programmers, but people who use XP in the prescribed way and for the right kinds of projects generally have more success than people who do not.

Typical of these happy campers is Robert Sartin, director of engineering at Impart Technology Inc. (formerly Terrace Mountain Systems), Austin, Texas. Impart has created a software platform based on Bluetooth to address the issue of peer-to-peer networking for mobile devices.

Sartin said he spends about 10% of his time in management and planning tasks, and the rest of the time developing code. Impart's eight-person shop has used "pure" XP—that is, all 12 practices—since day one, he said. "I initially decided to use XP because our requirements were very undefined and rapidly changing" and XP seemed to be able to handle that, he explained.

Initially, three of the first four programmers had a very strong background in "heavy process," and they wanted to continue using it. "My main argument against this approach was that since we did not know the requirements, we needed to use a process that supported exploring, and rapidly changing, the requirements," Sartin said. "Ultimately, by doing XP and demonstrating its agility in the presence of these changing requirements, we were able to overcome these fears."

He also had to help convince the Impart board of directors that XP was the route to go. "They had some issues, primarily regarding planning and reporting. We overcame those by involving management in the planning, so they could report directly to the board," Sartin said. The development team was very disciplined about reporting its velocity—that is, how many tasks they had accomplished and how far they still had to go for any given release—so that everyone had a realistic date for the next release of the software.

Sartin reported that XP mostly "eliminates costs" because he no longer needs to use "a heavy project planning system or design tool. I have found that simple tools like xpcgi for planning and JRefactory for modifying code and producing UML diagrams work well with little learning curve."

Another satisfied XP customer is the development team at Evant Solutions Corp. in San Francisco. Using XP, Evant has developed a suite of integrated inventory planning, forecasting, merchandise and inventory management, and decision-support applications. Rob Nee is director of engineering at the company.

Evant, too, is doing "pure" XP, a decision that Nee said was made by the CTO. Although most of the people at the company were behind that decision, Nee said it has led to some conflicts. One of XP's principles is that every developer adhere to a 40-hour workweek, so that everyone will last through the end of the project, do their best work and not burn out or leave.

At Evant, that work limit led to some trouble, Nee remembered. "We actually wound up working 50-hour weeks, but for a Bay Area Internet startup, people thought we were being ridiculous. They were saying if we can be this productive in 50 hours, what could we do in 80?" It took a while for some of the executives to grasp the concept, Nee recalled, laughing. "I think that the product's quality proved the degree to which we were predictable, and good for the long haul."

Edward Hieatt, lead engineer at Evant, recalls his initial resistance to XP. "I hadn't done it before, and [I] read a book by Kent Beck before my interview here. To be honest, I wasn't sure it was going to work. But over three or four months, it became apparent that it really does."

Some larger shops have used XP with mixed success—some of the most legendary XP projects had the plugs pulled before they were done. Among these are Chrysler's C3, Ford's VCAPS and a mainframe-to-PDF project at Interface Electronics Inc., Duluth, Ga. All were doing XP, more or less, and all got canceled for a variety of reasons. (The XP people involved claim that the decisions to cancel the various projects had to do with politics or corporate goals changing and nothing to do with technology or XP.)

Don Wells, a member of both the C3 and VCAPS teams and now an XP consultant in Sterling Heights, Mich., noted, "XP projects are particularly vulnerable to being cancelled, because you have running code at all times."

Still, those early projects proved XP's worth and value, according to those involved. Many members of Chrysler's C3 team have been spread around the company in various management jobs, and now there are about a dozen XP-related projects going on at DaimlerChrysler, according to Christian Wege, a Web app architect for the company based in Stuttgart, Germany.

Although they are not using pure XP, he said, "it is possible to introduce XP concepts into a large organization by interpreting given processes in a creative way." For example, they are using a detailed design document, because that is what current corporate process demands, but also pair programming and other XP elements.

Before the company was sold, Interface boasted one of the largest XP teams to date—32 programmers. Rob Whall, a member of the Interface team and now an independent XP consultant in Detroit, remembers being "very skeptical when I heard we were building the team. I didn't think that it would scale—but then we did it, not once but twice." They brought in groups of eight to 10 new programmers and paired the newbies up with experienced people. After about a month, everyone was up to speed.

Another practice at Interface was peer pressure to get everyone to write unit tests. Programmers had to mark, on a publicly displayed card, how many tests they wrote for how many methods. "We'd kind of kid around about who released the junky code—and what were they thinking?" recalled Tom Kubit, another former Interface employee and now a consultant based in Toledo, Ohio. "Everyone laughed and understood—it took a little cajoling, but eventually they all got it."

Today, other large companies are still using some XP policies and practices. (See the sidebar "Taking XP to the bank.")

Distributed XP?
The fact remains, however, that XP usage in the largest corporations remains more the exception than the rule. One issue, particularly these days, is that XP is intended for projects where everyone is working out of the same physical building. At its core, XP is mostly about a means of communicating among programmers, managers and the customer; its framework is meant to be used to keep everyone in the loop at all times, and sets things up so that everyone is contributing his or her best work to the process. To do that, XP maintains, people need to be in the same building—the same office, if possible. XPers talk about the need to rearrange cubicles and other work spaces to accommodate and reinforce the strong team aspects of the methodology.

Increasingly, however, distributed development is becoming a fact of life at most companies that are trying to attract and retain talented techies who may not want to or be able to come into an office every day. While XP may yet turn out to be suitable for far-flung teams, there is not enough heavy-duty experience with it to know for sure.

Confusing matters is that while some XPers frown upon the very notion of doing XP in a distributed manner because it runs so counter to current known and successful practices, others concede that perhaps the time has come to at least try it.

Don Wells suggested that instead of just assuming that distributed development is a way of life, "instead, maybe we should ask why teams need to be distributed. Will a team of fewer people, working together as a real team, build a better product than a team of geniuses working around the globe?" Many XPers echo that sentiment. Even those who are open to trying XP in a distributed fashion, however, concede that some crucial elements of communication will likely be lost.

The project rescuer
Overall, XP folks suggest trying the methodology first for a small project that is not mission-critical, training programmers, managers and customers about how it works, then ramping up to bigger projects. And if the suggested project is already in trouble, so much the better for ensuring its success, they say.

"I haven't been on a project where they said, 'Let's do XP' from the beginning," Wells said. "It's only been suggested on projects where they were doing poorly and needed help."

Such was the case with the VCAPS project, Ford's de Guzman recalled. "There wasn't a whole lot to lose" in trying XP, because the project was so far behind already. "It was a small risk," she remembered.

Jeffries cautioned, "It's dangerous to remove anything from XP, because we have already removed anything we did not think was necessary and that was safe to remove. In spite of the process being called 'extreme,' it's not about being stupid. It's about going really fast and really well, through great care and readiness. It's not about running as fast as you can—it's about being in condition and holding the racquet properly, which makes all the difference."

Kent Beck put the issue this way: "XP is a solution in search of a problem. It's really a nerd-oriented culture that wants to produce superior business value and have fun doing it. But if you're not prepared for the huge social shift, don't start this. On the other hand, maybe the gain is worth the pain."