In-Depth

Process Is the Rationale for Open-Source Development

• By Alan Radding
• December 1, 2005

The Big Idea

Method to Prevent Madness

• Development methodology and process entered the open-source world after IBM decided to contribute a subset of the IBM Rational Unified Process to Eclipse.
• Developers who have balked at the heavy-handed imposition of methodologies will find the new approach, at least in Eclipse's community, definitely more appealing.
• IBM's RUP announcement boosts Project Beacon, which was formed to create an extensible framework along with tools for software process engineering.

Just mention application development methodology, and watch the eyes of many developers and even veteran development managers glaze over. This is especially true in the open-source world, where the latest battle in the ongoing struggle to impose order on often freewheeling, ad hoc app dev initiatives has begun.

For many developers and their managers, development methodology and process conjure up images of suits barging in with thick three-ring binders bulging with forms, templates and checklists. It raises the specter of the tedious ISO 9000 and Six Sigma fads that burdened developers with endless meetings and paperwork, seemingly without helping them produce a single line of code.

Yet, development methodology and process are back, targeting the open-source world this time. In October, IBM announced it would contribute a subset—about 15 percent—of the IBM Rational Unified Process to the Eclipse open-source community, itself created from an IBM code contribution. RUP consists of a software process platform with a collection of methods and best practices for promoting quality and efficiency throughout software development projects. IBM expects its contribution to "provide a foundation architecture and Web-based tools for the industry to engineer, collaborate on, share and reuse software development best practices," according to IBM's announcement.

To some, the IBM announcement is just the latest sign of the growing maturity of the open-source app dev movement. If the Eclipse organization accepts the IBM contribution, pretty much a foregone conclusion, the RUPcontent will become the core of a major Eclipse initiative, the Eclipse Process Framework. Opensource proponents hope RUP and EPF will nudge enterprise software development organizations off the sidelines and get them to embrace open source wholeheartedly, rather than informally, the way many approach it now.

To others, the real story is not so much about the RUPcontribution—in fact, early comments on the Eclipse message board complained that 15 percent of RUP didn't amount to much—but about SPEM, OMG's Software Process Engineering Metamodel, and methodologies in general. SPEM enables methodology-based tools to share models and metadata. Still others saw the RUPcontribution, EPF and SPEM as the open-source community's longawaited response to Microsoft, which has been pushing its Microsoft Solution Framework as a simplified version of SPEM.

"With MSF, Microsoft had thrown down the gauntlet," says Ellen Gottesdiener, principal consultant, EBG Consulting and author of Requirements By Collaboration. "IBM had to respond. Fifteen percent [of RUP] isn't very much, but if RUP can be slimmed down and made practical, this is good," Amonth after IBM offered up the piece of RUP to the opensource community, Ivar Jacobson Consulting, which had signed onto the IBM announcement, announced it was teaming up with Microsoft to provide the Essential Unified Process. Essential UP is a streamlined and lightweight alternative to RUP, based on the Microsoft Solutions Framework and integrated with Microsoft Visual Studio 2005 Team System.

RUP's soft landing
IBM reports RUP is used by 500,000 developers around the world on projects ranging from small-scale product development to large industrial-strength systems. In practice, most RUPadopters tend to be large enterprises, usually within the Global 1000, which probably has to do with the hefty price tag and steep learning curve associated with deploying the full package.

"I don't see this having much impact, at least the way it is being presented initially," says Bob Bickel, VP of strategy at JBoss, an open-source middleware provider. RUP, and its use-case driven approach, already has become the de facto standard among large enterprise and government development organizations, he points out. "There will be some opensource programmers who will try to use it," he suspects, "but most of those who would use it probably already have it."

Others expressed some interest but weren't exactly rushing to jump on an open-source RUP bandwagon. "This looks like a bunch of modeling tools and plug-ins to Eclipse. We already have integrated tools and modeling, but we don't use a formal methodology," says David Glende, CTO of Unify, a business process automation provider. The company uses a blend of open-source and proprietary tools and spends a large percentage of its development time making sure the various tools work together. Unify, however, expects to "need a more formal process as we move into packaged vertical solutions," Glende concedes, but at the moment he is more interested in some of the Microsoft offerings.

Overall, the open-source message boards have been surprisingly quiet about the announcement. "I've noticed an incredible lack of buzz in the community about this," says Doug Houseman, associate director, Capgemini, the global consulting firm and one of the participants in IBM's initial open-source RUP announcement. It may be, he speculates, "that there are just a lot of loners in open source." Houseman hopes interest will pick up "once people realize how much it can help them." Capgemini uses its own methodology, Integrated Architecture Framework, at the early planning stages of the process. When it comes time to actually build software, "we bring in RUP and turn it into code," he notes.

"Is this announcement earth-shattering?" asks Richard Soley, chairman and CEO, Object Management Group. "No." What it will do as it gains momentum, he says, "is accelerate the interoperability of tools through SPEM, which is a good thing all around." More important than IBM's RUP announcement alone is the boost it gives to the EPF, also referred to as Project Beacon, of which RUPwill play a central part.

The goal of Project Beacon, which can be considered the second generation of SPEM, is to create an extensible framework along with tools for software process engineering. These tools will address process authoring, library management, and configuring and publishing a process. In addition, OMG expects Project Beacon to enable extensible process content that addresses a range of software development and management processes. These include iterative, agile, XP and incremental development and can be applicable to a broad set of development platforms and applications.

Given that this is all about high-level process and frameworks, it is not surprising the majority of open-source code jockeys aren't chiming in.

Open source is only part of what's needed
Software development organizations need something like RUP because the quality metrics for application development can be discouraging, to say the least. Fifteen to 20 percent of all software defects reach customers, costing the U.S. economy $60 billion a year, according to the Standish Group. Other industry analysts report that nearly half of internally developed software projects run over budget, 90 percent are completed late and 30 percent are canceled.

The problem starts long before the first line of code is written, maybe as early as the point when people are wondering if a new application is even needed. "When you come down to it, methodology is really a thinking tool, and before organizations start to code applications, they need to think," Gottesdiener says. And the more complex the application, the more thinking they need. Not only that, but requirements often don't get communicated effectively or at all.

The nature of the open-source community, where developers often work solo on their small contribution to a much bigger piece, exacerbates the communication problem. "Take a large distributed opensource project where you have communities of volunteer programmers. You chop up the project into pieces of work and pass the pieces and the information around. And then you lose people or have new people come in," Houseman says. Situations like these, which are not unlike those found in large enterprises, hinder effective communications. "Anything you can do to structure the sharing of information is welcome," he adds. Under such conditions, developers might indeed welcome structured methodology, process and modeling.

In its October RUPannouncement, IBM described just that kind of situation: "Today's software development organizations are increasingly global in nature and operate in a fluid environment where new teams are formed frequently, and developers are often shifted among teams as needs dictate. They must deliver quality applications that meet business goals, satisfy customers, and adhere to time and budget requirements."

Hindering them is not a lack of programming skills, but trouble with all the other things that must accompany development, such as consistent requirements setting, analysis and design, testing and project management. Typically, organizations try to bang together processes on the fly. One group of developers, however, doesn't know what worked with another group of developers, even within the same company. Documentation is left for later, and critical information necessary to meet compliance mandates can be overlooked.

Framework to capture practices
The many developers who have balked at the heavy-handed imposition of methodologies that dictate how they must develop software will find the new approach to methodology and process, at least in Eclipse's open-source community, definitely more appealing. "Our goal is to provide an infrastructure to collaborate around," says Per Kroll, manager of methods at IBM's RUP group. "We're not saying we know how open-source people should develop their software. We're providing a framework that captures what works for them and lets them share it. It's a meta model to capture best practices. Then we seed it with initial content," he continues.

The IBM contribution addresses requirements capture and definition, analysis and design, testing and project management. "We have an approach for how to take a requirement and describe it, implement developer tests and create an incremental build," says Kroll. Since it is based on RUP, it will surely emphasize the use of use cases. Most of it will take the form of XMLguidelines with attachments. Much of it will address best practices. The material will be available at Eclipse.

IBM expects the Eclipse community to expand upon the core content it has contributed. It also doesn't expect developers to adopt all of it. "A company may only want 30 percent of it, just the part that fits them," Kroll explains. Each development organization can pick and choose the pieces it takes based on what it needs and its own style.

"We have used a number of processes," says Kirti Vaidya, senior director at Covansys, which specializes in integration services. "Lately, it has been RUP. In the past, we thought of RUPas a methodology. Now we think of it more as a framework, and you can pick and choose what you need."

The bazaar and the cathedral
Number Six Software, a software consulting firm, is an early adopter of IBM's EPF contribution and has contributed some of its own content around process authoring. "We beta tested the process authoring tools with some of our customers," says Frank Dupont, practice manager. These first efforts focused on classic RUP tasks, such as definitions of roles and activities. "We have used it with customers on the requirements process—identifying actors and use cases. The tool helps you to author the specific steps, how you actually do the activity," he explains.

Once the activities have been identified and the steps specified, Dupont puts them into a work-breakdown structure that can be imported into a project management tool. As the project progresses through planning, design and modeling, other tools come into play. "The Beacon [SPEM] part lets you define everything using a meta model based on OMG. Once you capture it in the model, it can be integrated into other tools," he says.

Developers have a wide choice of tools. Kroll is quick to assure them that IBM's contribution contains no hidden hooks to lock an organization into RUP or the Rational tool set. "There are no direct links to ClearCase or any of our tools. We have separate guidances for our own products. This is technology independent," he insists.

To underscore the openness of the IBM contribution, the company lined up a diverse group of a dozen or more companies to participate in the initial announcement. These include Capgemini, BearingPoint, Covansys, Number Six Software, Ivar Jacobson International, Armstrong Process Group, Ambysoft, Object Mentor and Bedarra Research Labs, as well as Unisys, NTT Comware, Sogeti, Wind River, Jaczone and Object Management Group.

"We see this as more of Project Beacon," says Bobby Soni, senior VP and CTO of client services at Bearing Point.

The consulting firm, which has long used UML for large projects, sees the emergence of more process-centric development as an important trend leading to the development of composite business applications. In the latest IBM initiative, "we see evidence of these two tracks, UML and process-centric development, converging in process modeling," she says.

Although formal methodologies and processes may be new to the open-source community, other organizations, and not just large organizations, have come to rely on them with great success. "We grew from a one-person IT organization to 270 people in less than 2 years. We were bringing in new people so fast, people who we had to get up to speed. At the same time, the CIO and other top managers wanted us to standardize fast. RUP was the way to go," says Victor Certuche, software development lifecycle architect at Aegis Mortgage.

The company uses RUP in conjunction with a variety of products from other vendors, including Compuware and Borland (TogetherSoft). Aegis may have licensed and paid for conventional RUP, but Certuche still applauds IBM's contribution of pieces of RUP to the open-source community, believing it will benefit his organization as well. "First, it will create general awareness of RUP. Second, it will make more people familiar with RUP, which will increase the pool of skilled talent. Third, it will increase the availability of content," he explains.

Certuche hopes he can take advantage of contributed open-source plug-ins, specifically as the company tries to address compliance and security issues. Like it or not, more formalized methodology and process are coming to open-source app dev.

In the ongoing tension between "The Cathedral and the Bazaar," the title of Eric Raymond's seminal paper that kicked off the open-source development challenge to traditional software development approaches, the bazaar (the open-source community) is moving toward more structure while the cathedral (the big, structured enterprise) is loosening up a bit. In the end, every organization follows some kind of development process, even if it is only version control through a tool like CVS, which at the very least forces developers to check in and check out code.

"Every organization has to follow some kind of process when they develop software. The issue then becomes whether you document what you do so you can make it repeatable and predictable. Even a small organization needs a process for better communication," Dupont says. If the EPF evolves as expected, organizations may find they can pick and choose among a wide range of process and methodology pieces, all open source, to assemble and tailor a process, however idiosyncratic, that works for them.

ILLUSTRATION BY NIGEL SANDOR