Do tools matter?

• By Jack Vaughan, Michael W. Bucken
• February 1, 2004

The hammer stands as one of the greatest tools of all time -- it is still a welcomed companion in many a toolbox today. But its history is not unspotted. Recall Ned Ludd who famously took to hammering his workbench in the early 1800s.

That kind of love-hate relationship endures today in software development tools. Some bench-level developers are not slow to criticize tools as unwanted. Many can do more with simple command line development software (“Give me emacs or give me death!”), they say. In the days when hiring programmers was nearly impossible, programmers held more sway. Then, as now, development managers look for tools that can ultimately reduce project costs, maintenance costs and time to market.

In recent years, tools have taken something of a back seat to languages, platforms and technology. The Java language was a significant innovation, and it in turn affected the course of Microsoft technology and the development of .NET. In both camps, tools played catch-up. Often the end product was an app server platform -- “off-the-shelf middleware” -- that adhered to one component standard or another. Technology trends such as Web services caught fire, way ahead of actual implementation, not to mention tools.

The application development world was a little different in the past. Corporate and individual developers cut their teeth with TurboPascal, Visual Basic and so on. An era of client/server computing was built on the back of Gupta, Notes and a vast array of competitive offerings. Chief among the players there was Powersoft, a company that rose to software industry prominence on the merits of its PowerBuilder tool. (Today, it is part of Sybase.) The Delphi tool was somewhat late to the client/server tool party, but its developer, Borland Software, is one of the few toolmakers to survive into a new platform generation.

As the Internet penetrated business, more than one company tried to position its product as the PowerBuilder of the Web, but the biggest new product of the Web era -- BEA’s WebLogic app server -- depended, at least in the early going, on third parties for tools support.

Analysts and developers in the 1980s and 1990s might find the right methodology or 4GL tool, and then go off looking for problems to solve. Clearly, making the right choice of tool was essential to any successful project.

A new tech-savvy generation has now come along that is totally unencumbered by historical knowledge and not particularly overwhelmed by tools. Toolmakers are trying many approaches to convince developers that tools can significantly cut the complexity of projects, including the creation of standards that ensure tools can easily be replaced when better ones come along.

“Two initiatives that are very important in the world of app dev are Eclipse and the JTC [Java Tools Community],” said Gary Barnett, a research director at London-based Ovum Ltd. and a contributor to ADT. “The problem is that these [standards efforts] are effectively competing with each other, which leaves Microsoft’s Visual Studio in the position of being the most complete and highly integrated development studio on the market, something that should make supporters of Java and OSS very uncomfortable indeed.”

As the IT world finishes digesting the “Harvard Business Review’s” loaded question of the year, “Does IT matter?” [see “Harvard Experts put hurt on IT” by Jack Vaughan, ADT, July 2003], it is fair, it seems, to ask: Do tools matter?

Yes, said Anne Thomas Manes, an analyst at The Burton Group, Midvale, Utah. “Systems are becoming much more complex. Tools hide that complexity. A human can’t ‘grok’ all the different things that are going to influence a distributed computing application. So tools are more important now than ever before.”

“Yes,” said Adam Bosworth, chief architect and senior vice president of advanced development at BEA Systems, San Jose, Calif. “Tools always matter. Without Turbo Pascal and dBase, the creation of applications on the PC would have been immeasurably slower, and without VisiCalc people wouldn’t have gotten excited about PCs.

“Without Visual Basic, Windows wouldn’t have seen enormous numbers of apps built to take advantage of it and without Excel, customers wouldn’t have cared,” he noted.

“Systems programmers can and do live without tools,” said Bosworth. “They can write their own, live in the command line, and revel in complicated and arcane knowledge. But most people need and want tools to help them do their jobs.”

“I think some tools certainly matter, defining ‘matter’ as ‘let us deliver better software to our customers faster and cheaper,’” said Mike Gunderloy, an independent industry consultant in the Microsoft world and an ADT contributor. He sees new traction, in fact, for code generation tools. “Turning out all of the code needed to call every stored procedure in a database in two minutes instead of two days is a big win, even if you then have to go back and tweak 10% of the code,” said Gunderloy.

State of best-of-breed tools
Industry experts have proclaimed the death of best-of-breed-tools through the years, especially between the death of older technologies and the emergence of breakthrough ones. Many analysts and consultants argued periodically that technology growth had slowed and that the major platform players -- like IBM, Digital Equipment Corp., Sperry and Burroughs at one point, and Microsoft, IBM, Sun Microsystems, Oracle, Hewlett-Packard and others today -- would soon be selling the bulk of development tools and technologies to corporate IT shops.

And for a time, those experts appeared to be right. For example, before the emergence of client/server technologies, giants like IBM and Digital made moves to sell CASE and other development tools for their respective platforms via single-platform architectures the respective firms could control. However, the emergence of client/server technologies in the late 1980s pretty much killed the business of tools for building host-based apps. Fast-moving, third-party toolmakers like Powersoft, Gupta, Borland and others brought out tools for building the client/server apps now craved by corporate executives. But just a few years later, by the mid-1990s, the World Wide Web took over and led to another slew of start-ups building tools for that paradigm-shifting platform.

“There will always be room for innovation” and thus for a third-party development tool industry, said Bob Bickel, vice president of strategy and corporate development at Atlanta-based JBoss and a veteran of both the traditional platform and start-up worlds. Bickel speaks from experience as a key technology guru at J2EE app server pioneer Bluestone Software, where he headed the team that developed the Bluestone app server, which broke out from a very crowded field before its sale to Hewlett-Packard. HP abandoned the technology after an unsuccessful effort to take on IBM and BEA, but that is a story for another day and does not take away from the early success of the technology.

On the other hand, Bickel believes that open-source technologies can make it difficult for third-party firms and platform vendors to introduce new development platforms to the corporate IT world. “The cost of creating tools, and the emergence of open-source tools will make it increasingly difficult to justify the creation of new development environments,” said Bickel, who is now responsible for spreading the open-source JBoss app server into the corporate mix.

“Borland will continue to be important because of the large installed base and the new innovation they are bringing to the market. Also, there will be a need to increase the productivity of distributed development teams as outsourcing and open- source development play an ever-increasing role in how projects are developed,” he added.

“If my toolbox only contained tools from Microsoft,” added Microsoft-centric consultant Gunderloy, “it would be pretty darned empty. Just off the top of my head, the tools I depend on in daily development include CodeWright [now owned by Borland], XMLSpy [Altova], NUnit [open source], NDoc [open source], FinalBuilder [AtoZed Software], FogBugz [Fog Creek], CityDesk [Fog Creek] and Vault [SourceGear]. In some cases, these tools are going head-to-head against Microsoft and winning, at least on my desktop; XMLSpy blows away the native XML editing features of the Visual Studio .NET IDE, and SourceGear Vault is so much better than Visual SourceSafe that there’s almost no comparison.”

Gunderloy said Microsoft still tries to help third-party tool vendors by “providing an increasingly better experience for best-of-breed tools to integrate. XMLSpy is a good example; it uses the Visual Studio Integration Program APIs to provide me with seamless editing of XML files, within Visual Studio .NET, using the XMLSpy tools. It all just works. XMLSpy also has a standalone version, but other vendors concentrate on extending VS .NET to fill in its holes; for instance, CodeSmart adds an integrated view of files and classes that Microsoft could have written but didn’t. At least on the Microsoft side of the house, the platform vendor is laying down the foundation and leaving it to ISVs to build a large part of the superstructure.”

Gunderloy does note that third-party success does depend a great deal on the platform vendor’s philosophy toward ISVs -- competitor or partner. “For this strategy to succeed, the platform vendor has to make it easier to integrate than to ship separately,” Gunderloy said. “Microsoft has gone a long way in Visual Studio .NET, but there’s still room for improvement. Their source code control APIs, for example, are pretty much a dog’s breakfast.”

Open-source tools
Note that Gunderloy mentions open-source development tools in the same breath as offerings of independent software suppliers. The growing acceptance of open-source technologies like the Linux OS, Apache app server and others is expanding into development shops for a variety of reasons, according to JBoss’ Bickel, whose firm distributes and sells support for the open-source JBoss Web server.

“Open source is having a profound impact on corporate developers by dramatically lowering costs, improving efficiency and productivity, and increasing quality,” contends Bickel. “Open source lowers costs because software components that once had to be paid for or developed in-house are now freely available. Open source improves efficiency because it eliminates the need to build many components and tools in-house.”

Bickel also maintains that open-source tools will build higher quality applications. “Open source increases quality because open-source projects tend to have on average higher quality [components] than individual components built in-house. For example, developers used to routinely create their own build infrastructures -- now it is common to use [the open-source] Ant, which is much more functional and stable than what many developers had created to solve their own problems in a custom way.”

Other observers maintain that developers should use the best tool for a project regardless of its internal technology. “Open-source software touches on development in so many ways that it’s impossible to come up with a single answer” as to how it will affect corporate development operations, said Ovum’s Barnett. “Open-source development tools are, superficially at least, attractive because they appear to lower the cost of development. But the reality is that the cost of tooling up is a tiny fraction of the overall cost in development. So my advice is still to pick the tool that offers the greatest productivity in terms of development services [the IDE] and its integration with the other tools you will [or should] be using -- modeling tools, testing tools, configuration management tools.”

Unsurprisingly, JBoss’ Bickel said open-source integration technologies are changing the world of middleware. “Open source is having a huge impact on the middleware market. It has become the dominant Web server. The issues of enterprise-level support and viability seem to be addressed by the new professional open-source business models that companies like JBoss and MySQL are built on.”

Aspect-Oriented Programming
Bickel points to an emerging programming technique, Aspect-Oriented Programming (AOP), as a potential boon for developers. “The next big step technically for middleware will be a movement toward ease of development,” Bickel said. “The emergence of AOP will ease the complexities developers have with programming models like J2EE, Web services and messaging.”

AOP is described in an IBM research paper as a method that allows programmers to “modularize crosscutting concerns,” which are depicted as behavior that cuts across the typical divisions of responsibility, such as logging.

Bickel said developers and toolmakers are already utilizing AOP’s advantages. “We think AOP will become very pervasive,” he said. “It is embedded in what Microsoft is doing. We think it will take another 12 to 18 months for the Java community to establish some standards in terms of frameworks, interfaces and common aspects.”

BEA’s Bosworth added that AOP may be familiar to many developers. “I’m not convinced that AOP is indeed that new given that VMs [virtual machines] have long allowed intrusive interception,” he said. “Threaded Interpretive Languages [TILs] were written up by Byte magazine in the very early 1980s for this ability. But it is promising and we’re starting to look hard at how we can take advantage of it.”

Meanwhile, Uche Ogbuji, a consultant and co-founder at Boulder-based Fourthought Inc., sees AOP as a minor add-on feature for object-oriented programmers. “I think AOP is a minor concession in OO circles that trying to code all object behaviors imperatively leads to horrid maintenance tangles,” Ogbuji said. “It is probably a big revelation to anyone who has drunk too deeply of the OO Kool-Aid, but a very minor and unremarkable nuance to developers who have maintained a broader world view.”

JBoss’ Bickel suggested that developers can utilize and gain immediate advantages from the method. “Early adopters will prove this technology very useful and greatly improving productivity. Once standards emerge, it will take off and create a major change in the middleware market.”

Meanwhile, Linux may or may not become pervasive in IT back rooms and/or on the desktop, but observers say the OS by itself should have little effect on the day-to-day responsibilities of corporate developers.

“Most tools and development environments are separate from the OS -- one of the reasons why it is simple to move software onto Linux,” Bickel said. “One area that may prove interesting for Linux environments is infrastructure elements like security, management and middleware that are becoming available to make the environment richer for infrastructure and enterprise applications.”

In addition, said Bickel, “The other area is in the potential of using Linux as a desktop alternative to Microsoft,” a notion that many in the Microsoft community say is a pipe dream, whether talking about Windows or its successor, Longhorn, which is still two or three years away.

“The truth is that Linux on the desktop gets way too much attention,” said consultant and ADT columnist David Chappell. “Linux is still struggling to reach the Mac desktop market share, yet nobody talks about Longhorn being a Mac killer.”

Java and .NET ease of use
Java tools have been a focus of industry effort for more than five years now. But only a few toolmakers have supported themselves solely on such tools. As enterprise software elements and, of late, Web services standards have been added to the J2EE infrastructure, the Java developer’s job has become bigger.

But -- smart as they are -- Java developers’ brains have not necessarily grown bigger as well.

The Java language, and its blood brother, the J2EE component and architecture standard, did a great deal to focus attention away from tools and toward the glue and artifacts the tools leave in their wake -- what you might call middleware. In fact, Java app servers have gained attention at the cost of tools until quite recently.

The deliberate efforts of BEA, Borland, Compuware, IBM, JetBrains, Sun and others have been directed toward taking some complexity out of that job (see “In search of a gentler Java,” ADT, Dec. 2003). Sometimes these vendors appear intent on targeting the “blue-collar” Visual Basic developer ranks that comprise one of the biggest communities of tools users. Meanwhile, Visual Basic developers are undergoing some rockin’ and rollin’ as they migrate to the new .NET development model.

The next big step technically for middleware in general will be a movement toward ease of development said JBoss’ Bickel. For his part, he sees the emergence of the AOP paradigm as a step toward easing the complexities developers have with programming models like J2EE, Web services and messaging.

Educating IDEs
A common comment through the ages is that you have to know how to use the tools you are given. Tool education will continue to be a key, said Steve Garone, chief analyst and managing partner at The AlignIT Group in Boston. (See More from Garone elsewhere on this web site.) And a recurring message in J2EE education, he said, is that with Java deployment tooling is often more critical to success than development tooling.

“If Java is going to be more important as services-based models proliferate, it’s not surprising that we see activity around tooling,” said Garone. “Our research shows that, with the exception of very small companies [those that likely do not have to deal with a great deal of complexity in their IT environments], the education of developers is one of the most significant barriers to adoption of IDEs.”

Ease of deployment and time to deploy are two areas of great concern to those using IDEs, said Garone. Vendors who are part of the new JTC effort -- IBM is on the outside of JTC to date -- are likely trying to address three issues, according to Garone:

• Making it easier for enterprises to leverage their IT personnel more efficiently by creating easy-to-use tools that abstract away technologies with which they do not have experience;

• Creating a more "IT-friendly" development scenario; and,

• "Countering both Microsoft and IBM and their efforts to dominate in the development arena."

In an effort to reduce complexity, 2004 will be the year companies start taking a serious look at tools that offer alternatives to code-centric development, said Bob Barker, vice president of strategic planning and technology at Compuware Corp. “As the Java skills gap becomes critical, Compuware sees the model-driven, pattern-based [MDPB] development going mainstream,” he said, adding that “we have been preaching MDPB since before it was ‘cool.’”

Java servers: Middleware on the side
While the Java app server has undone the status quo of years ago, full standard off-the-shelf middleware packs seem as distant a dream as ever. There is always someone wanting to add something to the mid-tier stack. The Burton Group’s Manes has described Web services as the “mighty morphing platform.

There are portals, authentication services, Web service message queue schedulers, orchestrators, dispatchers, BPM and BAM. All bear watching for managers -- and all need better tooling if they are ever to become everyday product enhancements. Many move deep into the realm of business logic. “Another aspect of the mighty morphing platform is that more and more code is getting pushed down into the bowels of the platform, so that developers don’t have to write as much,” said Manes.

“BPM may be the next big thing -- the rise of Service-Oriented Architecture makes BPM much more attractive -- but it certainly isn’t a big thing yet,” said consultant Chappell. To ease BPM implementation, business process execution languages have been devised. The state they are in is still nascent.

“It’s clear that people expect way too much from BPEL. Given the amount of platform-specific logic that business processes typically have, portable BPEL processes are probably a pipe dream. And when you remember that BPEL is focused on defining what’s required for interoperability, not portability, this shouldn’t be too surprising,” said Chappell.

.NET-Redmond campfires
Java tool trends have been of interest in no small part due to the occasional contention between vendors. Such contention is far more muted in the Microsoft tools camp, where the main questions are what is in the next release of Visual Studio .NET and Windows. A slew of code-names are involved.

Microsoft is adding better modeling traits [see “Modeling for .NET,” ADT, January 2004] and a whole lot more to Whidbey, its next tools rev, and improving its XML support in Yukon, its next SQL Server rev. Much anticipation focuses on the Longhorn OS, as was indicated by the vigorous turnout at Microsoft’s Professional Developers Conference in Los Angeles late last year. C#, which takes a few pages out of Java’s play book, is certainly gaining more interest all the time.

If Longhorn is late, will it matter? We asked David Chappell, who turned our question on its ear. “Longhorn can’t possibly be late, since Microsoft hasn’t announced a release date. Clever strategy, that,” noted Chappell.

Although a desktop phenomena, Longhorn sets the stage for server futures. “Longhorn is, in fact, a desktop operating system, but there’s so much stuff that’s now going into the desktop operating system, so part of Longhorn is Avalon, which is the Windowing user interface/user experience piece,” said The Burton Group’s Manes.

“And there’s also Indigo, which is the whole middleware piece, and then there’s Whidbey, which is the development tool piece, and Yukon, which is the SQL Server piece. So you’ve got all these different pieces that are part of this Longhorn solution, but the first release of Longhorn will be the desktop system. The next release after that, which has a different code name, will be the server operating system, but it will all be based on the Longhorn operating system,” said Manes.

An interesting trend to look for: Any incursions easier-to-use Java language tools can make in the Visual Basic developer community. BEA, in particular, has not been shy to note that it is tailoring its WebLogic Workshop tools for that end-market. As Java, theoretically, gets simpler, it is not clear that .NET is not getting more complex. Some VB developers have had their hands full migrating to that architecture over the last two years.

Modeling
Ease of use has a lot to do with auto-generation of code and modeling -- two big topics these days and, of course, ongoing topics throughout the history of development tools. UML-keeper OMG has spawned some of this interest as it has promoted the Model Driven Architecture (MDA) as a means to prepare for the day when tools for Java and C# give way to new tools and languages.

“People are starting to see the value of tools that offer a high level of abstraction, said Compuware’s Barker. “The bottom line is that the model-driven and pattern-based development engine will continue to rev. The productivity boost companies realize by automatically generating significant portions of applications speaks for itself.”

At the same time, said Barker, the increasing complexity of business-critical applications and the dearth of highly experienced J2EE developers is a real opportunity for tool providers to improve developer productivity and ramp-up developers of all skill levels building enterprise Java applications.

But developers are not enamored completely with models. OMG head Richard Soley wryly notes a typical software project can well contain a “model burning” step, staged by the development team. What bothers them? Among other things, that the code and model are never quite in sync.

“Models that are disjoint from the code tend to fail,” said BEA’s Bosworth. “The code evolves, morphs and is debugged under stress and soon deviates so much as to be unrecognizable.”

Bosworth said BEA’s model for easing J2EE system building is deliberately not about code generation. “It is about making the plumbing easier. All the business logic and app logic is described either declaratively [like JSPs describe pages and JPDs describe Business Processes] or procedurally -- for example, in code -- or as extensions to the declarative model,” he said.

“This [type of] model is a tried and true one and is at the heart of PowerBuilder, VB and other successful frameworks,” he added.

Models are the basis of software development, and the importance of modeling cannot be overemphasized, said Fourthought’s Ogbuji. “Even in using agile techniques, such as XP, developers are still building a model, although much less formally.

“A rapid prototype can serve as an informal model. The main problem with informal models is that they are not very reusable, and they do not contribute to maintenance of apps,” added Ogbuji.

“Good programmers know this in their hearts, but the reason they express a desire to burn models is that modeling tools and, in some cases, methodology, impose a lot of unnecessary overhead. All a model need do is allow the developer to express clearly the real-world concepts being addressed in the application,” he explained.

The useful model can be as simple as a well-conceived data dictionary and need not automatically lead, asserted Ogbuji, to “analysis paralysis that people associate with MDA and other OMG initiatives.”

But, Ogbuji admits, “if the concepts being modeled are complex enough to require the rigor of MDA, then just punting with RAD is a sure-fire way to doom a project to failure.”

Overall, however, Ogbjui thinks “code generation from models is a pipe dream and always will be.” Which indicates perhaps that a love–hate relationship with software tools may go on through time, just as it does with tools in other spheres of endeavor.

In the end, successful development requires some modest care and commitment to the model, and talented developers to translate that to high-quality code. “There is no automating around this fact,” said Ogbuji.

Please see the following related stories:
“Software applications infrastructure: The new middleware?” by Steve Garone
“Tool talk” by Staff