Process Is the Rationale for Open-Source Development
- By Alan Radding
- December 1, 2005
The Big Idea
Method to Prevent Madness
- Development methodology and process entered the open-source world after IBM decided to contribute a subset of the IBM Rational Unified Process to Eclipse.
- Developers who have balked at the heavy-handed imposition of methodologies will find the new approach, at least in Eclipse's community, definitely more appealing.
- IBM's RUP announcement boosts Project Beacon, which was formed to create an extensible framework along with tools for software process engineering.
Just mention application development methodology,
and watch the eyes of many developers and even veteran
development managers glaze over. This is especially
true in the open-source world, where the latest
battle in the ongoing struggle to impose order on often freewheeling,
ad hoc app dev initiatives has begun.
For many developers and their managers, development methodology
and process conjure up images of suits barging in with thick
three-ring binders bulging with forms, templates and checklists.
It raises the specter of the tedious ISO 9000 and Six Sigma fads
that burdened developers with endless meetings and paperwork,
seemingly without helping them produce a single line of code.
Yet, development methodology and process are back, targeting
the open-source world this time. In October, IBM announced
it would contribute a subset—about 15 percent—of the IBM
Rational Unified Process to the Eclipse open-source community,
itself created from an IBM code contribution. RUP consists
of a software process platform with a collection of methods and
best practices for promoting quality and efficiency throughout
software development projects. IBM expects its contribution to
"provide a foundation architecture and Web-based tools for the
industry to engineer, collaborate on, share and reuse software
development best practices," according to IBM's announcement.
To some, the IBM announcement is just the latest sign of the
growing maturity of the open-source app dev movement. If the
Eclipse organization accepts the IBM contribution, pretty much
a foregone conclusion, the RUPcontent will become the core of
a major Eclipse initiative, the Eclipse Process Framework. Opensource
proponents hope RUP and EPF will nudge enterprise
software development organizations off the sidelines and get
them to embrace open source wholeheartedly,
rather than informally, the way
many approach it now.
To others, the real story is not so much
about the RUPcontribution—in fact, early
comments on the Eclipse message
board complained that 15 percent of RUP
didn't amount to much—but about SPEM,
OMG's Software Process Engineering
Metamodel, and methodologies in general.
SPEM enables methodology-based tools
to share models and metadata. Still others
saw the RUPcontribution, EPF and SPEM
as the open-source community's longawaited
response to Microsoft, which has
been pushing its Microsoft Solution Framework
as a simplified version of SPEM.
"With MSF, Microsoft had thrown
down the gauntlet," says Ellen Gottesdiener,
principal consultant, EBG Consulting
and author of Requirements By Collaboration.
"IBM had to respond. Fifteen
percent [of RUP] isn't very much, but if
RUP can be slimmed down and made
practical, this is good," Amonth after IBM
offered up the piece of RUP to the opensource
community, Ivar Jacobson Consulting,
which had signed onto the IBM
announcement, announced it was teaming
up with Microsoft to provide the Essential
Unified Process. Essential UP is a
streamlined and lightweight alternative to
RUP, based on the Microsoft Solutions
Framework and integrated with Microsoft
Visual Studio 2005 Team System.
RUP's soft landing
IBM reports RUP is used by 500,000 developers
around the world on projects
ranging from small-scale product development
to large industrial-strength systems.
In practice, most RUPadopters tend to be
large enterprises, usually within the Global
1000, which probably has to do with the
hefty price tag and steep learning curve associated
with deploying the full package.
"I don't see this having much impact,
at least the way it is being presented
initially," says Bob Bickel, VP of strategy
at JBoss, an open-source middleware
provider. RUP, and its use-case driven
approach, already has become the de facto
standard among large enterprise and
government development organizations,
he points out. "There will be some opensource
programmers who will try to use
it," he suspects, "but most of those who
would use it probably already have it."
Others expressed some interest but weren't
exactly rushing to jump on an open-source
RUP bandwagon. "This looks like a bunch
of modeling tools and plug-ins to Eclipse.
We already have integrated tools and modeling,
but we don't use a formal methodology,"
says David Glende, CTO of Unify, a
business process automation provider. The
company uses a blend of open-source and proprietary
tools and spends a large percentage of
its development time making sure the various
tools work together. Unify, however, expects
to "need a more formal process as we move
into packaged vertical solutions," Glende
concedes, but at the moment he is more interested
in some of the Microsoft offerings.
Overall, the open-source message boards
have been surprisingly quiet about the announcement. "I've noticed an incredible lack
of buzz in the community about this," says
Doug Houseman, associate director, Capgemini,
the global consulting firm and one of the
participants in IBM's initial open-source RUP
announcement. It may be, he speculates, "that
there are just a lot of loners in open source."
Houseman hopes interest will pick up "once
people realize how much it can help them."
Capgemini uses its own methodology, Integrated
Architecture Framework, at the early
planning stages of the process. When it comes
time to actually build software, "we bring
in RUP and turn it into code," he notes.
"Is this announcement earth-shattering?"
asks Richard Soley, chairman and CEO,
Object Management Group. "No." What
it will do as it gains momentum, he says,
"is accelerate the interoperability of tools
through SPEM, which is a good thing all
around." More important than IBM's RUP
announcement alone is the boost it gives
to the EPF, also referred to as Project Beacon,
of which RUPwill play a central part.
The goal of Project Beacon, which can be
considered the second generation of SPEM,
is to create an extensible framework along
with tools for software process engineering.
These tools will address process authoring,
library management, and configuring and
publishing a process. In addition, OMG
expects Project Beacon to enable extensible
process content that addresses a range of
software development and management
processes. These include iterative, agile, XP
and incremental development and can be
applicable to a broad set of development
platforms and applications.
Given that this is all about high-level
process and frameworks, it is not surprising
the majority of open-source code jockeys
aren't chiming in.
Open source is only part
of what's needed
Software development organizations need
something like RUP because the quality
metrics for application development can be
discouraging, to say the least. Fifteen to
20 percent of all software defects reach
customers, costing the U.S. economy $60
billion a year, according to the Standish
Group. Other industry analysts report that
nearly half of internally developed software
projects run over budget, 90 percent are
completed late and 30 percent are canceled.
The problem starts long before the first
line of code is written, maybe as early as the
point when people are wondering if a new
application is even needed. "When you
come down to it, methodology is really a
thinking tool, and before organizations start
to code applications, they need to think,"
Gottesdiener says. And the more complex
the application, the more thinking they
need. Not only that, but requirements often
don't get communicated effectively or at all.
The nature of the open-source community,
where developers often work solo on
their small contribution to a much bigger
piece, exacerbates the communication
problem. "Take a large distributed opensource
project where you have communities
of volunteer programmers. You chop
up the project into pieces of work and pass
the pieces and the information around.
And then you lose people or have new people
come in," Houseman says. Situations
like these, which are not unlike those found
in large enterprises, hinder effective communications.
"Anything you can do to
structure the sharing of information is
welcome," he adds. Under such conditions,
developers might indeed welcome structured
methodology, process and modeling.
In its October RUPannouncement, IBM
described just that kind of situation: "Today's
software development organizations
are increasingly global in nature and operate
in a fluid environment where new teams are
formed frequently, and developers are often
shifted among teams as needs dictate. They
must deliver quality applications that meet
business goals, satisfy customers, and adhere
to time and budget requirements."
Hindering them is not a lack of programming
skills, but trouble with all the
other things that must accompany development,
such as consistent requirements
setting, analysis and design, testing and
project management. Typically, organizations
try to bang together processes on the
fly. One group of developers, however,
doesn't know what worked with another
group of developers, even within the same
company. Documentation is left for later,
and critical information necessary to meet
compliance mandates can be overlooked.
Framework to capture practices
The many developers who have balked at
the heavy-handed imposition of methodologies
that dictate how they must develop
software will find the new approach to
methodology and process, at least in
Eclipse's open-source community, definitely
more appealing. "Our goal is to
provide an infrastructure to collaborate
around," says Per Kroll, manager of
methods at IBM's RUP group. "We're
not saying we know how open-source
people should develop their software.
We're providing a framework that captures
what works for them and lets them
share it. It's a meta model to capture best
practices. Then we seed it with initial
content," he continues.
The IBM contribution addresses requirements
capture and definition, analysis
and design, testing and project management. "We have an approach for how to
take a requirement and describe it, implement
developer tests and create an incremental
build," says Kroll. Since it is based
on RUP, it will surely emphasize the use of
use cases. Most of it will take the form of
XMLguidelines with attachments. Much
of it will address best practices. The material
will be available at Eclipse.
IBM expects the Eclipse community to
expand upon the core content it has contributed.
It also doesn't expect developers
to adopt all of it. "A company may only
want 30 percent of it, just the part that fits
them," Kroll explains. Each development
organization can pick and choose the
pieces it takes based on what it needs and
its own style.
"We have used a number of processes,"
says Kirti Vaidya, senior director at Covansys,
which specializes in integration services.
"Lately, it has been RUP. In the past,
we thought of RUPas a methodology. Now
we think of it more as a framework, and
you can pick and choose what you need."
The bazaar and the cathedral
Number Six Software, a software consulting
firm, is an early adopter of IBM's EPF contribution
and has contributed some of its
own content around process authoring. "We
beta tested the process authoring tools with
some of our customers," says Frank Dupont,
practice manager. These first efforts focused
on classic RUP tasks, such as definitions of
roles and activities. "We have used it with
customers on the requirements process—identifying actors and use cases. The tool
helps you to author the specific steps, how
you actually do the activity," he explains.
Once the activities have been identified
and the steps specified, Dupont puts them
into a work-breakdown structure that can be
imported into a project management tool.
As the project progresses through planning,
design and modeling, other tools come into
play. "The Beacon [SPEM] part lets you define
everything using a meta model based on
OMG. Once you capture it in the model, it
can be integrated into other tools," he says.
Developers have a wide choice of tools.
Kroll is quick to assure them that IBM's
contribution contains no hidden hooks
to lock an organization into RUP or the
Rational tool set. "There are no direct links
to ClearCase or any of our tools. We have
separate guidances for our own products.
This is technology independent," he insists.
To underscore the openness of the IBM
contribution, the company lined up a diverse
group of a dozen or more companies
to participate in the initial announcement.
These include Capgemini, BearingPoint,
Covansys, Number Six Software, Ivar
Jacobson International, Armstrong Process
Group, Ambysoft, Object Mentor and
Bedarra Research Labs, as well as Unisys,
NTT Comware, Sogeti, Wind River,
Jaczone and Object Management Group.
"We see this as more of Project Beacon,"
says Bobby Soni, senior VP and
CTO of client services at Bearing Point.
The consulting firm, which has long used
UML for large projects, sees the emergence
of more process-centric development
as an important trend leading to the
development of composite business applications.
In the latest IBM initiative, "we
see evidence of these two tracks, UML
and process-centric development, converging
in process modeling," she says.
Although formal methodologies and
processes may be new to the open-source
community, other organizations, and not
just large organizations, have come to rely
on them with great success. "We grew from
a one-person IT organization to 270 people
in less than 2 years. We were bringing in
new people so fast, people who we had to
get up to speed. At the same time, the CIO
and other top managers wanted us to standardize
fast. RUP was the way to go," says
Victor Certuche, software development
lifecycle architect at Aegis Mortgage.
The company uses RUP in conjunction
with a variety of products from other vendors,
including Compuware and Borland
(TogetherSoft). Aegis may have licensed and
paid for conventional RUP, but Certuche
still applauds IBM's contribution of pieces
of RUP to the open-source community, believing
it will benefit his organization as
well. "First, it will create general awareness
of RUP. Second, it will make more people
familiar with RUP, which will increase the
pool of skilled talent. Third, it will increase
the availability of content," he explains.
Certuche hopes he can take advantage
of contributed open-source plug-ins,
specifically as the company tries to address
compliance and security issues. Like it or
not, more formalized methodology and
process are coming to open-source app
In the ongoing tension between "The
Cathedral and the Bazaar," the title of Eric
Raymond's seminal paper that kicked off
the open-source development challenge
to traditional software development
approaches, the bazaar (the open-source
community) is moving toward more structure
while the cathedral (the big, structured
enterprise) is loosening up a bit.
In the end, every organization follows
some kind of development process, even
if it is only version control through a tool
like CVS, which at the very least forces developers
to check in and check out code.
"Every organization has to follow some
kind of process when they develop software.
The issue then becomes whether
you document what you do so you can
make it repeatable and predictable. Even
a small organization needs a process for
better communication," Dupont says. If
the EPF evolves as expected, organizations
may find they can pick and choose
among a wide range of process and
methodology pieces, all open source, to assemble
and tailor a process, however idiosyncratic,
that works for them.
ILLUSTRATION BY NIGEL SANDOR