In-Depth

Open-source users find rewards in collaborative development

• By Alan Joch
• February 1, 2005

Banks pride themselves on playing things close to the vest. So, when Dresdner Kleinwort Wasserstein’s IT group shared the code of OpenAdapter, an internally developed tool, with the development community, it caused a sensation. “It was astonishing,” recalls Steve Howe, DrKW’s global head of open-source initiatives. “They found it difficult to believe a bank was open sourcing something.” OpenAdapter, a back-end Java integration tool that helps integrate bank apps with little or no custom programming, had become “a famous piece of software within DrKW,” Howe says. “Half of the decision to open source it into the wider financial community was to try to replicate that enthusiasm.”

DrKW’s motive for releasing the tool was hardly altruistic. By releasing the tool to the wider financial community, the investment bank hoped to benefit from the bug fixes and refinements other programmers in Europe and North America might make.

However, Howe and other open-source experts warn that corporations’ code-sharing projects require a number of safeguards to protect intellectual property and keep companies from becoming unwitting victims or distributors of destructive code. Some companies believe such risks outweigh the potential benefits and resist open-source business applications. DrKW and other firms believe careful open-source collaboration gives them a competitive advantage.

Open source matures
Open source is no longer relegated to backroom projects. IDC recently predicted Linux would capture more than 20 percent of the server market this year, with a growth rate about twice that of Windows. Many corporations now consider open-source alternatives when they evaluate technologies for new projects. “Open source is attacking the underlying costs of running an IT department,” Howe reasons. “The lower our infrastructure costs, the better.”

Open source’s popularity is twofold. First, IT managers believe paying little or nothing to license open-source software gives their resource-strapped departments a financial edge on new initiatives. In addition, open-source software often provides the capabilities to meet requirements without burdening IT systems with the feature bloat that commercial software vendors sometimes rely on to distinguish themselves. “With our code, we found it ran better and the costs were less with JBoss [an open-source application server] than with a commercial alternative,” reports Raven Zachary, director of Internet technologies for the La Quinta Inns hotel chain, Irving, Texas. “That’s probably because JBoss is more lightweight than the very robust, feature-rich commercial stack.”

Collaborative development is another big advantage of the open-source model because a large group of interested developers often provides extra eyes and ears that can be extensions of in-house development departments. Now that such advantages have opened the open-source door into corporate IT departments, managers are turning their attention to these advantages to determine how best to incorporate the revisions and innovations open-source developer communities create, while using in-house resources to tailor products to individual needs.

Collaboration considerations
Open-source success requires more than just the freewheeling, collaborative attitude frequently associated with the open-source movement, says Jim Herbsleb, an associate professor at Carnegie Mellon University’s International School of Computer Science and a member of its Institute for Software Research. After studying several open-source development efforts, including that of the hugely popular Apache Web server, he believes several conditions must exist for open-source development to thrive.

Open-source techniques in commercial development work best when programmers maintain or enhance existing software, not when they develop the software’s initial core functionality, he says. “In commercial software development, the early stages of design work are highly collaborative; people spend a lot of time together in meetings,” Herbsleb explains. “In open-source development, that kind of face-to-face contact is exactly what you don’t have.” Instead, open-source communications rely on e-mail messages and forums.

Herbsleb also observes people who are most active in developing open-source software are typically those who will become power users of the programs. Problems arise when end users range beyond this group, such as in a corporation in which non-technical business people are the target users of a new application. “Doing the extra work to understand the needs in some depth of an end-user community is not something that’s usually a part of open-source development,” he says.

Finally, open-source development works best in a kind of IT autocracy. “The rhetoric around open -source focuses on democracy and group participation,” Herbsleb says. “That’s certainly an aspect, but I’ve found that most successful open-source projects are run with an iron fist.”

A single person or a small group should control what which code goes into new releases, he says. “If someone offers code that the group thinks is inappropriate for whatever reason, they keep it out.”

The core developers manage the features definitions of a new application, assure code quality, and set deadlines. In some cases, companies have two collaboration communities: one a tightly regulated group of core developers, the other a more public group with wide participation that primarily tests the code to identify bugs. The first group is also responsible for corporate reputation, combing through code looking for anything damaging or unlawful.

Organizations that can establish this overall development structure can reap benefits from high-quality, easily modified code without licensing fees. “It’s like finding a big stash of computers for free,” Herbsleb says.

Banking on open source
DrKW believes it found development success with OpenAdapter by creating a variation on the collaborative model the open-source community pioneered. The integration tool uses SpiritSoft’s SpiritWave Enterprise Service Bus to facilitate communications among applications that must share and integrate data. “We open sourced the tool within DrKW and told people that if they needed to change OpenAdapter slightly, they were free to do that,” Howe says. “A lot of developers decided they could make a name for themselves by contributing to the software.” Today, more than 100 applications in the bank use the integration tool.

To tap similar expertise outside DrKW, the bank turned to CollabNet, a commercial project-development platform from CollabNet in Brisbane, California. CollabNet provides a number of services, including version control tools and discussion forums common in free open-source clearinghouses such as SourceForge. DrKW went with a commercial solution because it wanted to tap the company’s open-source expertise. CollabNet helped the bank establish a separate legal entity called the Software Conservancy, which owns the intellectual rights to OpenAdapter.

The site publishes the latest stable build of the software, which anyone can download. Users can also send in complaints and error reports and suggest bug fixes. So far, DrKW has received input from developers at competing banks and from companies outside the financial industry. Collaboration consists of technical subjects, not information that involves trade secrets or confidential customer data.

Developers interested in becoming more involved in OpenAdapter’s evolution can gain access to the source code by signing a legal agreement that assigns the intellectual property rights of their code to the Software Conservancy. About 20 developers now hold such status.

Legal waivers aren’t DrKW’s only safety net. Before making its code public, the bank combed through developer comments that referenced IP addresses and host server names that may have made it easier to break through DrKW’s security defenses. It stripped out the standard developer comments that might prove embarrassing or even libelous. “We edited out things like, ‘I wrote this code much faster than Martin because I’m a better developer than Martin is,’” Howe says.

The bank relies on its own in-house testing processes rather than bug reports from the larger community to assure OpenAdapter’s safety. “We try to unit test everything. As we write the code, we write tests for the new code,” Howe says. In addition, after a code freeze, the company spends another two to three weeks testing the latest build. “When we do a new release, we want to make sure it really works.”

All of these considerations mean that while open-source software doesn’t carry licensing fees, it’s not necessarily free. “Open-source isn’t free of responsibilities,” Howe says. “Even when someone sends you a bug fix, you have to test to make sure there really is a bug, and, if there is, that the fix actually works. Open source isn’t software for nothing. You have to make an investment in it.”

So far, that investment has paid off with OpenAdapter. Howe believes the input he gets from the open-source community is like having a couple extra development people on his staff to improve and refine the code. There’s another intangible benefit. “This project has improved the reputation of DrKW, especially among younger developers,” Howe explains. “We’ve been able to attract a lot of talented people who are interested in doing open-source development. This makes DrKW a cool place to work in compared to other investment banks.”

Reining in the code
Lucent Technologies/Bell Labs, in Naperville, Ill., is developing a multimedia server for telecommunications companies based on the maturing Session Initiative Protocol standard. Three years ago, Lucent began embracing the open-source collaboration model to revise the server, using company developers. “As SIP gained momentum, we saw the need to have a unified Lucent approach, and that’s where precepts of open source come in,” explains Vijay K. Gurbani, author of the server and a Lucent technical staff member.

Confirming Herbsleb’s observations about collaboration success, Gurbani confines source code changes to a small collection of core developers who work in groups assigned to various portions of the server. Given his status as the original developer, Gurbani remains responsible for the server’s core architecture and guides decisions about which new features should go into the server. Others within Lucent contribute performance enhancements, compression capabilities, and changes in related standards. “I act as the clearinghouse for what tasks need to be done,” Gurbani says.

The developers use the Web to communicate and coordinate tasks, but so far, Lucent hasn’t set up a structure as formal as what’s possible with SourceForge or CollabNet. “We haven’t reached the point where anybody can download source code from the Web,” Gurbani explains. “It’s currently mostly a manual task: If a new [group of developers] wants the code, I send it to them via e-mail or post it on a Web site.”

By keeping tight reins on the source code, Gurbani can direct the server’s evolution. “Some developers may want to change the code in ways that I feel are not conducive to the architecture as a whole and I’ll decide not to include it,” he explains, adding anyone is free to customize their own versions. “The changes may make sense in their world but not to the general version of the product,” he says.

Deciding what goes in and what stays out isn’t entirely autocratic, however. “In a pure open-source environment, it’s the guy who’s in charge who gets to decide,” Gurbani says. “But here, we’re driven by the needs of customers who ultimately pay us for our technology. Because of that, the decisions become more of a management than a technical issue. The choices are driven by me and by managers who talk with customers to determine which new features should be added right away and which ones might be nice to have but can wait.”

This collaboration of technical and managerial input results in a better product, Gurbani believes. “My research interests lay in signaling technologies, not in performance enhancement, so when I write code, I’m thinking about what I need to do to make the server compliant with standards. But when I turn the code over to our Bell Labs colleagues, who are performance experts, they say, ‘You could have used this technology here to the server run faster.’ Another group might say that adding compression will improve things. I’d love to add that, but alone I just don’t have the time.” With the open-source approach, innovations don’t have to wait, he says.

Cautious approach
Nevertheless, even companies that embrace collaborative open-source development for back-end systems say they’re cautious when it comes to business applications. La Quinta Inn’s Internet apps, which allow customers to reserve rooms via the Web, rely on a who’s who of open-source technologies, ranging from Apache Web servers to TomCat JavaServer pages and JBoss application servers. A member of La Quinta’s IT staff actively contributes to the TomCat code base. However, Zachary says the company’s business applications are proprietary and too linked to legacy systems to risk sharing with anyone outside the company. “There’s not an opportunity to contribute our code in some open-source initiative,” he says. “We might contribute code if, in our development cycle, we find a bug in an open-source infrastructure product like JBoss or TomCat. Then we’re happy to contribute changes for common good. But otherwise, I don’t think our company, for intellectual property reasons, would want to contribute core business code into the open-source community.” Some things will always remain close to the vest.

Discussion Points
Use Discussion Board below

OPEN-SOURCE COLLABORATION
• Collaborative development of open-source software can yield cleaner code and more innovative applications.
• Two challenges: Protecting intellectual property and guarding against the possibility of unknowingly distributing destructive code.
• Open-source development may work best in autocratic IT environments. Jim Herbsleb of Carnegie Mellon University says the most successful projects he’s seen are run with an iron fist.