'Extreme' Method Simplifies Development Puzzle
- By John K. Waters
- January 1, 2001
New lightweight XP methodology stresses teamwork, testing, planning and
simplicity; uses real-world experience as design patterns for a new generation
of software developers.
The words "extreme programming" conjure up visions of thrill-seeking programmers
bungee jumping into projects without a plan, of caffeine cowboys jamming code
through the night with nary a document to guide them, and of daredevil developers
working without a net. In fact, the principles and practices that comprise this
emerging software development methodology, known to its relatively small but
expanding circle of devotees as XP, is, in many ways, about as controlled as
you can get.
According to Uttam Narsu, senior analyst at Cambridge, Mass.-based Giga Information
Group, XP is what is known as a lightweight software development methodology.
Lightweight methodologies have few rules and practices, all of which are fairly
easy to follow. In contrast, heavyweight methodologies, which most software
developers currently embrace, involve elaborate rules, extensive up-front planning
and piles of ongoing documentation — what Narsu calls the "artifacts of the
"[XP] is reminiscent of the patterns movement," noted Narsu, "which comes
out of the Smalltalk world. It embraces the notion of being able to abstract
real-world experiences of software architects into design patterns that guide
people who don't necessarily have years of experience. It basically says, 'Look,
lots of people have been developing software for many, many years. They have
evolved a series of practices that work. Let's use those practices.'"
Ward Cunningham, one of the fathers of XP, agrees: "Extreme Programming is
a lot of simple little things. It's lots of things that have been done before,
some of which have even been discredited. We have reconstructed a collection
of practices that support each other in a way that is startling."
Cunningham is a computer consultant well known for his work in object-oriented
programming practices. He is one of the "Three Extremos" whose work and thinking
helped produce this methodology. His Web site, Wiki Wiki Web, has become the
de facto online home of XP.
Computer consultant Kent Beck is the Extremo generally credited with assembling
XP practices for the first time and naming the methodology. He did it almost
on the fly in 1996 when he was brought in as a consultant on a floundering project
at Daimler-Chrysler. Chrysler wanted to replace a group of disparate payroll
apps with a single appli-cation. Officially called the Chrysler Comprehensive
Compensation project, it is known in XP lore as "C3." Beck took over the project,
dumped everything that had been done before and started from scratch.
"In the course of two or three weeks of interacting with the Chrysler people,
I made it all up," said Beck. "If you look at it, there's really nothing new
there. I guess you could say that it was putting the pieces together. I would
literally stand up in front of [a white board] with a marker in my hand and
not know what I was going to say. It was an exciting time."
Beck describes XP as a "programming-centric" methodology. "It says that the
thing we as programmers do that adds value to the business is write code," he
explained. "Everything else is window dressing, and we should do as little of
everything else as possible. Documentation, design, formal review, separate
QA; it's all a waste of our time. [XPers] do what we do very intensely, but
we don't do a bunch of the other stuff."
Testing: The heartbeat of XP
Testing has long been the ugly stepchild of software development, but Cunningham
describes it as the "heartbeat" of the XP methodology.
"Instead of doing big design up front, we write tests up front," he noted.
"Every day before you write code, you write a test to see whether that code
works. It's the practice that prevents chaos. It is through constant testing
that an XP project yields a very high level of certainty that the program is
working. It gives developers a kind of freedom you don't find in other methodologies."
XPers test like maniacs throughout the development process. They perform unit
tests for each class as they code. All unit tests in the system must be running
at 100 percent before any developer releases his or her code. The C3 project
reportedly had more than 1,300 unit tests.
XPers also perform functional tests of each feature defined by a user story.
Unit tests check the behavior of a single class; functional tests check the
system from end to end. On XP projects, developers build a continually growing
collection of functional tests, which they run automatically, usually overnight.
XP proponents recommend that functional tests be run daily.
"The idea," said Beck, "is that the developers are responsible for proving
to their customers that the code works correctly; it's not the customer's job
to prove that the code is broken. From a customer's standpoint, it's all about
transparency, risk reduction and the ability to adapt to change. And the customer
gets real measures of what is actually going on in the project, not someone's
opinion. They get demonstrated progress. Fifty percent done of an XP contract
means that 50 percent of the features are 100 percent complete."
Of user stories and pair programming
A cornerstone of the XP process is the user story. A user story is a short
(about three sentences), plain English description of what the customer wants
the software to do, written by the customer. Each feature of the application
is defined by an individual user story. They are printed out or, more often,
hand-written on cards, sort of like a Hollywood scriptwriter's storyboard. From
these user stories, XPers estimate how long a project will take, establish priorities
and develop functional tests.
User stories take the place of the large requirements documents typical of
heavyweight methodologies, and they differ strikingly from traditional requirements
specs in their level of detail; they are short and to the point. When it is
time to implement the story, the developer and the customer sit down together
and expand on the original description.
According to Beck, a typical XP project of six to 12 months calls for between
50 and 100 user stories. If there are too few stories, it becomes difficult
to define priorities; if there are too many, the whole thing becomes unwieldy.
Individual user stories are broken down into engineering tasks. If it looks
like an engineering task is going to take more than a week to complete, that
task is broken down further.
In addition to providing the user stories, customers play an ongoing role in
the XP process. For all intents and purposes, the customers are part of the
team. They must be readily available to answer questions, resolve disputes and
set small-scale priorities.
The XP methodology emphasizes teamwork, and works best with small teams of
between two and 10 programmers. The whole team owns the software, and anyone
can find him or herself working on any part of it. XP takes the team approach
to, well, extremes in a practice known as pair programming. In an XP project,
programmers double up, two to a workstation. XP proponents insist that two programmers
working side-by-side are more than twice as productive as one for a given task.
"Pair programming provokes talk between programmers about programming," explained
Cunningham. "As they struggle with the lack of language to describe what they're
doing, a vocabulary emerges. Pretty soon a vocabulary develops, a language that
is about the act of programming. And that's something that people have just
plain done without — to their detriment. This is an experience that people have
just not shared. But because we can talk about it, it forms in our minds a much
better mental model of programming."
According to Ron Jeffries (the third Extremo), many coders are at first resistant
to the idea of pair programming. "But once they try it," said Jeffries, "most
of them really like it. I'm not comfortable with the idea of pairing two junior
programmers, but you can put them together in just about any other combination.
Senior programmers — real superstars — find that working with an inexperienced
programmer makes them better. And the junior programmers get better much faster
when they work with an experienced [person]."
Jeffries worked with Beck on the C3 project, becoming XP's first "coach." In
an XP project, the coach oversees the project and makes sure that it remains
extreme. The coach maintains the project vision and communicates it to the other
team members. The coach also keeps individuals on track, egos in check and even
acts as a mentor when appropriate.
Planning the XP way
Their aversion to monolithic processes notwithstanding, XPers do plan. They
get past some of the pitfalls inherent in heavyweight methodologies with a process
called "the Planning Game" — which actually takes the form of a game, with goals,
playing pieces, players and moves.
The basic playing pieces are the user stories, which are written on cards.
The players are the customers and the developers, who get together and move
the cards around on a large table. The basic goal of the planning game is to
create a release plan that specifies which user stories are going to be implemented
for each release and the dates for those releases.
Once the release plan is set, customers and developers get together for iteration
planning meetings, which are held at the beginning of each iteration to produce
that iteration's plan of programming tasks. Customers choose from the set of
user stories that will be implemented during the next iteration. These selected
stories are then translated into individual programming tasks that will be implemented
during the iteration to complete the stories.
XPers plan individual iterations in detail, but not until just before each
iteration begins. (For the complete "rules" of the planning game, point your
browser to http://c2.com/cgi/wiki?PlanningGame.)
In its pursuit of a clean and spare overall design, the XP methodology also
employs something called a "spike solution," or just plain "spike." A spike
is a very simple program that XPers write to explore potential solutions to
tough technical or design problems. According to Cunningham, Beck coined the
term: "I would ask Kent [Beck], 'What's the rock-bottom simplest thing we can
program that will convince us we're on the right track?' That question takes
you outside the complexities within the existing code, which is probably distracting
you from the essence of the requirement. The idea is to write the smallest possible
code that could be said to perform a function independent of the existing mechanism."
In the XP methodology, spikes serve primarily to help a programmer work through
a problem; once the problem is sorted out, most spike solutions are thrown away.
According to Cunningham, it was the advent of object-oriented programming that
served as the impetus for the development of XP. "Everybody knew that OO had
a lot to offer," he said, "but people had a lot of trouble with it. Objects
are about modeling; modeling is about making judgments. With algorithms and
data structures you can sort of get by any which way. But to get objects to
work, things have to be written well."
So, how do you take the plunge into an XP project? There are basically three
ways to get started: provide your development team with immersion training before
you begin; hire a coach to work with you during the course of the project; or
as Beck put it, "Just rearrange the furniture and get going."
One of the few places — maybe the only place — to go for XP training is a company
called Object Mentor Inc. in Green Oaks, Ill. The company offers mentoring,
training and outsourced development services. Both Beck and Jeffries teach through
If you plan to jump in on your own, Beck recommends starting with the Planning
Game. "If you don't have the stories, or at least the beginning of the stories,
you risk wasting lots of time solving problems that aren't problems. I ask this
question a hundred times of a new XP team: 'What story are you working on?'
Generally the answer is, 'Well, none, it just sort of seemed this stuff needed
Cunningham suggests taking on one XP practice at a time to introduce your firm
to the methodology. Unit testing may have the most impact on the project, he
said, but he agrees that the Planning Game is invaluable. Unit testing has the
advantage of being relatively easy to adopt. Pair programming can be a good
place to start, too, he noted, but if your people are uncomfortable with it
and you have to harp at them, they will probably miss the point.
One of the fastest ways to ramp up an XP process is to hire an experienced
coach. "By the time you get to iteration four," Cunningham said, "you'll know
what you're talking about. It seems to take that long to get the kinks out."
Is XP ready for prime time?
Critics argue that XP and other lightweight methodologies differ little from
the style of programming that formal methodologies replaced. It is a step back,
they say, to the bad old days of unstructured development. Giga's Narsu dismisses
"I don't really think any of the XP people are saying, 'Gee let's throw away
design and just code,'" he said. "Many of the critics who make that kind of
argument don't really understand XP."
XP is not throwing out all the rules, explained Jeffries; it is simplifying
methodologies and eliminating practices that hinder the development process,
while hanging onto rules that work. "If you look at the situation," he said,
"almost no one is really following the heavy methodologies anyway. I'm seeing
people who seem to be moving instinctively toward XP." Jeffries describes XP
as "a backlash against heavy methodologies." The lightweight nature of the XP
methodology, added Beck, is a reaction to document-oriented software processes
that saw coding as little more than a clerical task, something that was done
after all the interesting work was finished.
"Many people in development are observing that [following heavy methodologies]
just doesn't work," said Beck. "XP is clean and concise, and it allows programmers
to be creative and productive while remaining organized and focused."
But XP does have its shortcomings; chief among them, said Giga's Narsu, is
its customer participation requirement. "In many of the real-world projects
that I've dealt with, the reaction is not terribly good among the stakeholders.
These folks have a lot to do, and the notion that they should be available to
clear up any problems with a user story ... well, it's not going to work in
Another issue Narsu is concerned with is the inherent limitation of the pair
programming practice in distributed development environments. "Can you really
do pair programming when one programmer is in Paris, Texas, and the other one
is in Paris, France?" he asked. "Even with the good collaborative tools available
today, is it really the same as sitting next to your partner?"
Beck sees the methodology's tendency to expose organizational weaknesses as
a potential downside of XP. "If your marketing folks are telling you what to
do every two weeks, it's obvious that they're clueless. Lots of people react
to that by huddling up and not communicating with [the developers]. The proper
XP response is to make it obvious that those decisions are not being made. You
should say that no programming projects can go forward until [those decisions]
are [made]. The responsibility has to go somewhere; on an XP project, it goes
where it belongs."
Giga's Narsu worries that XP may simply be too new to have all the kinks worked
out. "Keep in mind that we don't have much experience with this methodology,"
he said. "This is an evolving methodology. Is it ripe and ready? It's hard to
say at this point."
There are no hard statistics on the number of XP projects underway, but most
of those interviewed for this story believe that there are about 100 — a number
Jeffries describes as "insignificant in the grand scheme of things, but surprising,
given how resistant most people are to change."
Marty Thorne's Portland, Ore.-based company, the Client Server Group, is currently
utilizing the principles of XP on a project for Pacific Gas & Electric. He said
that the "extreme" moniker is a bit of a misnomer. "It's called 'extreme' because
XP sort of drowns out any other practice," explained Thorne. "An extreme project
is very stable and almost sedate. Because of the principles, you can move forward
fearlessly. You can change code at will. You're designing, but you're doing
it incrementally. And you only put in what you need."
Who wants to go to extremes?
Anecdotally at least, there seems to be a lot of interest in XP these days.
XP sessions at the Software Development (SD) 2000 conference held in San Jose,
Calif., in March were jammed with attendees. Marc Pardi, director of the conference,
said that the enthusiasm he saw at the XP sessions reminded him of SD's first
Java sessions. "[XP] has an almost 'cultish' following," he said. "And what's
driving that is the pressure on developers to get corporate e-commerce sites
up and running now. Even their corporate apps are driven by the Web. The developers
are the unsung heroes in this new economy. These guys need the tools, and XP
certainly seems to be one of them."
Although Beck takes credit for assembling the initial core principles of the
methodology during his involvement with C3, he allows that the process grew
from other methodologies, and that it is evolving in the hands of developers.
"It's sure not my baby any more," he said.
As for the name, Beck defends it vigorously. "It was exactly the right name
for it," he said, "because it takes what we do to extremes: everybody unit tests;
we build and test eight to 10 times a day, we code review constantly, we iterate
on a one- or two-week cycle. XP pushes its practices to the limit."
and principles of XP
(XP) is an evolving methodology that comprises a number of practices and
principles. What follows is a list of the 12 practices of Extreme Programming.
1) Unit Tests — unit
tests are written to run in batches and test classes. Each unit test typically
sends a class a fixed message and then verifies that it returns the predicted
2) Pair Programming
— the practice of teaming up two programmers per workstation is one of
XP's better-known techniques. According to XP proponents, studies have
shown that, once they get used to it, two programmers working side by
side are more than twice as productive as one programmer for a given task.
3) Refactor Mercilessly
— if you find two methods that look the same, you should refactor the
code to combine them. Two objects with the same functionality should be
refactored into one.
4) You Aren't Gonna
Need It — Ron Jeffries explained this XP practice this way: "Always implement
things when you actually need them, never when you just foresee that you
[will] need them." On an XP project, coders are discouraged from 'someday-we're-going-to-need-it'
thinking. By staying focused on today's tasks today, XPers avoid the waste
of time and resources that often accompany speculative side trips. XPers
implement only what they really need, not what they might need.
5) Functional Tests
— programs or scripts configured to test that packages (groups of clusters
of classes) meet external requirements and achieve goals such as performance.
6) Onsite Customer
— on XP projects, the customer must be readily available to the development
team, first to create the user stories, but later on to answer questions,
resolve disputes and to set small-scale priorities. Essentially, the customer
is part of the team.
7) Continuous Integration,
Relentless Testing — XP calls for frequent releases of tested code, a
practice called "continuous integration, relentless testing."
"You can't release
unless the code works," said Ward Cunningham. "Therefore, to have continuous
integration, you must have relentless testing." XP programmers write unit
tests for every class they create, and they perform frequent functional
tests, which, in the words of Jeffries, "...test the system from end to
end, input files to output files." The result, proponents say, is rapid
progress and a program that always works better than it did the day before.
8) Planning Game —
takes the form of a game, with goals, playing pieces, players and moves.
The basic playing pieces are the user stories, which are written on cards.
The players are the customers and the developers, who get together and
move the cards around on a large table. The basic goal of the planning
game is to create a release plan that specifies which user stories are
going to be implemented for each release and the dates for those releases.
Once the release plan is set, customers and developers get together for
iteration planning meetings, which are held at the beginning of each iteration
to produce that iteration's plan of programming tasks.
9) System Metaphor
— the XP word for architecture. A handful of classes and patterns that
shape the core flow of the business the system will implement.
10) Frequent Releases
— on his Web site (www.xprogramming.com), Jeffries talks about XP development
teams typically releasing their software at least once per day. He writes:
"We have found that when developers sit on code for more than a day, it
is usually because they are in trouble. When we understand what we are
setting out to do, it is almost always quite possible to break down the
work so that our changes can be released daily."
11) Collective Code
Ownership — In an XP project, the code belongs to the project, not to
the individual programmer. As the coders develop required functionality,
they may browse into and modify any class.
12) Coding Standards
— because people will come and go during the project, XPers strive to
use the same coding conventions in every regard.
In addition, Extreme
Programmers also live by the following two principles: Do The Simplest
Thing That Could Possibly Work. This is XP's prime directive, a fundamental
principle considered the most important contributor to the rapid progress
of XP projects. It is a mandate to implement a new capability in the simplest
way you possibly can, as long as it works. This idea infuses everything
They're Just Rules.
Proponents of XP are passionate about their methodology, but they are
nothing if not flexible. "The XP rules ... evolved as part of the process
of getting the C3 team from dead stop to full speed," said Jeffries. "But
these aren't my rules or Kent [Beck]'s rules. They are the rules that
the team embraces. We do require everyone to follow them, because we believe
that they make us more effective. When the situation changes, or we get
a better idea, we discuss possible changes and often change the rules.
After all, 'They're Just Rules' is one of our rules."
getting attention among software developers.
Crystal — the name
for a family of lightweight methodologies developed by Alistair Cockburn.
The Crystal methodologies are based on the idea that software development
is a cooperative game in which the participants help each other to reach
the game's goal: the delivery of software.
The Crystal approach
involves capturing the elements that make a project successful. It has
grown out of what Cockburn describes as "... a series of project debriefings
over half a decade ... on projects I have worked on directly, and in conversations
with other consultants visiting other projects."
In practice, Crystal
seeks to give development team members the most latitude possible. It
asserts that small teams produce the best results when using lightweight
methodologies combined with extensive person-to-person interaction and
feedback from the code. It also adheres to the principle that even the
simplest process constraints are generally too difficult for team members
to adhere to.
The latest version
of Cockburn's methodology is called Crystal Clear. Designed for four-
to six-person projects, it attempts to describe a full methodology of
the lightest kind that will still produce good results. On Wiki Wiki Web,
Cockburn writes, "The difference between Crystal Clear and Extreme Programming
is that XP is much more disciplined, [while] Crystal Clear is much more
tolerant, even sloppy-looking. I would say XP is more productive, Crystal
Clear more likely to get followed."
Scrum — Scrum is a
formal, rapid application development methodology for quickly developing
enhancements to existing software experiencing frequent shifts in requirements.
"Scrum" is not just an acronym (S-Coordinates Rutgers University Model),
but also a metaphor. The name derives from a formation in the game of
Rugby in which players surround the ball and move it forward together.
Scrum assumes that
all requirements are not known in advance, and that the fastest path to
surfacing and implementing all requirements will be discovered empirically
during the development process. Careful control mechanisms are used to
ensure on-time delivery of a high-quality product, while allowing maximum
flexibility of small, tightly coupled development teams. The Scrum methodology
requires a well-motivated team and good leadership to implement effectively.
C3 Project Ends
Despite its many virtues,
Extreme Programming (XP) cannot save every project. Recently, the fabled
Chrysler Comprehensive Compensation (C3) project, which had been humming
along for about three years, was cancelled. Kent Beck attributes the cancellation
to a change of developers.
"C3 was about consolidating
12 or 13 payroll systems into one," he explained. "The first [developer]
was very focused on that. But he left, and the new [developer] wasn't
so interested in that. The customer stories started not being about the
payroll system, and it stopped making progress. It points out that there
are things outside the scope of XP that can sink projects."