Columns

How does your AD effort measure up?

• By Jack Vaughan
• October 31, 2002

Development managers have been reeling like most everyone else in the business world. They are burdened by cutbacks in budgets and forced under great pressure to make tough decisions, based on economics, about which projects and people to keep on. In effect, they are asked to measure what they are worth.

Making such measurements can be seen as cold-hearted to those who view software development as an art. That said, measurement is hardly cool -- many projects during the Internet gold rush years got underway without a lot of planning. Counting code was ''un-cool.'' But measurement is a good thing to do. Being ready to explain the cost, risk and value of application development efforts means one had the foresight to take stock, and this is what the best development managers generally do. They regularly step outside the fray, go to first principles and measure their organization's strength. It's a big part of best practices.

Taking stock seemed to naturally arise as a key theme at Gartner's Application Development Summit held in mid-September in Orlando, Fla. After several years, an economic boom and bust, and a few bouts of new technologies, best practices and best processes are once again the best courses of action for beleaguered technology management. The Gartner conference armed its attendees to return to organizations where business managers tend to view application development departments as giant money holes.

Vortex to go
Of course the money hole mantle is simply unfair. Application development managers work hard -- they feel like producers, not like money-consuming vortexes. The label derives from an often willful misunderstanding of what AD does. But application development managers cannot react emotionally to that willfulness -- they must play the cards they are dealt.

The main focus of the Gartner sessions was on establishing valid processes for development. Emphasis was placed as well on ways of ensuring that there were compelling business cases for projects undertaken. There were even a few bits of evidence indicating that the ''AD Money Hole'' has something of the nature of a myth.

In recent years in U.S. businesses, application development has become far more effective and a lot less expensive, said Robert Solon, research director for the Gartner Measurement practice, speaking at the AD Summit. Solon continually studies the matter and trawls for information about what goes on in development.

''We were almost half as expensive in 2001 as [we were] in 1993. This is very interesting news. We're always told we're spending too much,'' said Solon, speaking to and for the application development legions.

Solon estimates that cost of development per function point (FP) has declined from $390/FP in 1993 to $254/FP in 2001. And the decline was pretty steady. This is a substantial improvement, especially when factoring in inflation. The year 2000 bug-fix effort did run counter to the prevailing cost-reduction trend, but that trend has reappeared since 2000.

Function point users unite
Like other Gartner analysts, Solon does not claim that function points are a be-all and end-all for measurement, but function points may be the most useful tool now available for estimating what application development does.

Function points, as described by the International Function Point Users' Group, establish a numeric index according to type and complexity. These indexes are totaled to give an initial measure of size, which is then normalized by incorporating factors relating to the software as a whole, to measure the overall size and complexity of a software product.

''Application development has controlled costs, especially in support,'' said Solon. While Gartner has good evidence to back up this assertion, Solon admits that AD managers are still going to get plenty of cost pressures in days to come.

While saving money, application development has improved its customer satisfaction ratings since 1997, estimates Solon. Time to market as expressed in FP/month has increased during the same timeframe. But Solon notes the increases seen in development speed of fresh projects are not reflected to the same extent in enhancement efforts devoted to existing applications. Here, documentation, one of the dreaded bugaboos of best practices, raises its head. Often, new development projects ''economize'' on documentation and leave it to the enhancement teams to figure it all out, Solon noted.

Solon said that application development must measure itself to effectively merge goals with its business-unit sponsors. Application development managers have to be able to truly say they know what type of an organization they have before jumping into the fire. Solon and other Gartner analysts at the AD conference pointed to the Capability Maturity Model (CMM) framework as perhaps the most useful categorization means at hand for establishing how mature an application development organization is. ''Knowing thyself'' in this admittedly somewhat arbitrary way helps a lot when it is time to make decisions on technology and other application matters, Gartner Group indicated.

Take Web services for example. While that integration technology has something of the new about it, it also has aspects familiar to anyone who has been at this trade for more than a few years. Thus, it was not jarring to discover that several Gartner analysts likened Web services to software components discussed widely since the mid-1990s. After all these years, the questions of what a component is and how one manages and promotes component reuse are still heard. Gartner's Matt Hotle, group vice president and research group director, was among the analysts who said that an organization that has not yet succeeded with components is not a good candidate to suddenly succeed with Web services. Only with measurable processes in place can the average organization make decisions on this technology and then follow through to reach a predictable, favorable goal.

Align, align, align
Gartner's Solon advised managers to integrate key performance indicators into their performance management systems to meet the goal of aligning application development efforts with business objectives.

And while Solon urged development leaders to contribute to corporate efforts to measure the benefits of application development to the business, he cautioned that such ''Return on Investment'' (ROI) questions can only be answered if the business side is truly ready to try to measure business benefit.

Application development management must understand the drivers of performance, said Solon, who groups the key input drivers of application development into four categories: personnel, environment, technology and processes (with software requirements counted as part of processes). The output of all this is software that should comprise a mix of cost, efficiency, quality and functionality goals. Measurement should be applied at various stages of the process. Solon chides, however, that measurement without action is dangerous. For one thing, when it becomes an ivory tower- or monastic-type discipline, it starts to become less accurate, or inaccurate.

As corporate competitors embrace new technologies and gain advantage, corporate management will be knocking on application development's door, seeking to replicate or surpass those offerings. The message from Florida's AD Summit was: ''Fine, but ... new technology is not the point.'' With proper practices in place, taking on new technology will be less of a crazy adventure. Proper practices here mean measuring -- taking time to take stock. CMM and function points may help, but those are just some of the many varied means. The important thing is to take the time to know the development organization as best as one can, while still getting the work done to spec on time.

What do you think? Is measurement more trouble than it is worth? Is an intuitive understanding of your team all that is needed? What about CMM or function points? Voice your opinion. Send an e-mail to Editor-at-Large Jack Vaughan at [email protected].