How does your AD effort measure up?
- By Jack Vaughan
Development managers have been reeling like most everyone else in the
business world. They are burdened by cutbacks in budgets and forced under great
pressure to make tough decisions, based on economics, about which projects and
people to keep on. In effect, they are asked to measure what they are worth.
Making such measurements can be seen as cold-hearted to those who view
software development as an art. That said, measurement is hardly cool -- many
projects during the Internet gold rush years got underway without a lot of
planning. Counting code was ''un-cool.'' But measurement is a good thing to do.
Being ready to explain the cost, risk and value of application development
efforts means one had the foresight to take stock, and this is what the best
development managers generally do. They regularly step outside the fray, go to
first principles and measure their organization's strength. It's a big part of
Taking stock seemed to naturally arise as a key theme at Gartner's
Application Development Summit held in mid-September in Orlando, Fla. After
several years, an economic boom and bust, and a few bouts of new technologies,
best practices and best processes are once again the best courses of action for
beleaguered technology management. The Gartner conference armed its attendees to
return to organizations where business managers tend to view application
development departments as giant money holes.
Vortex to go
Of course the money hole mantle is simply unfair.
Application development managers work hard -- they feel like producers, not like
money-consuming vortexes. The label derives from an often willful
misunderstanding of what AD does. But application development managers cannot
react emotionally to that willfulness -- they must play the cards they are
The main focus of the Gartner sessions was on establishing valid processes
for development. Emphasis was placed as well on ways of ensuring that there were
compelling business cases for projects undertaken. There were even a few bits of
evidence indicating that the ''AD Money Hole'' has something of the nature of a
In recent years in U.S. businesses, application development has become far
more effective and a lot less expensive, said Robert Solon, research director
for the Gartner Measurement practice, speaking at the AD Summit. Solon
continually studies the matter and trawls for information about what goes on in
''We were almost half as expensive in 2001 as [we were] in 1993. This is very
interesting news. We're always told we're spending too much,'' said Solon,
speaking to and for the application development legions.
Solon estimates that cost of development per function point (FP) has declined
from $390/FP in 1993 to $254/FP in 2001. And the decline was pretty steady. This
is a substantial improvement, especially when factoring in inflation. The year
2000 bug-fix effort did run counter to the prevailing cost-reduction trend, but
that trend has reappeared since 2000.
Function point users unite
Like other Gartner analysts, Solon
does not claim that function points are a be-all and end-all for measurement,
but function points may be the most useful tool now available for estimating
what application development does.
Function points, as described by the International Function Point Users'
Group, establish a numeric index according to type and complexity. These indexes
are totaled to give an initial measure of size, which is then normalized by
incorporating factors relating to the software as a whole, to measure the
overall size and complexity of a software product.
''Application development has controlled costs, especially in support,'' said
Solon. While Gartner has good evidence to back up this assertion, Solon admits
that AD managers are still going to get plenty of cost pressures in days to
While saving money, application development has improved its customer
satisfaction ratings since 1997, estimates Solon. Time to market as expressed in
FP/month has increased during the same timeframe. But Solon notes the increases
seen in development speed of fresh projects are not reflected to the same extent
in enhancement efforts devoted to existing applications. Here, documentation,
one of the dreaded bugaboos of best practices, raises its head. Often, new
development projects ''economize'' on documentation and leave it to the
enhancement teams to figure it all out, Solon noted.
Solon said that application development must measure itself to effectively
merge goals with its business-unit sponsors. Application development managers
have to be able to truly say they know what type of an organization they have
before jumping into the fire. Solon and other Gartner analysts at the AD
conference pointed to the Capability Maturity Model (CMM) framework as perhaps
the most useful categorization means at hand for establishing how mature an
application development organization is. ''Knowing thyself'' in this admittedly
somewhat arbitrary way helps a lot when it is time to make decisions on
technology and other application matters, Gartner Group indicated.
Take Web services for example. While that integration technology has
something of the new about it, it also has aspects familiar to anyone who has
been at this trade for more than a few years. Thus, it was not jarring to
discover that several Gartner analysts likened Web services to software
components discussed widely since the mid-1990s. After all these years, the
questions of what a component is and how one manages and promotes component
reuse are still heard. Gartner's Matt Hotle, group vice president and research
group director, was among the analysts who said that an organization that has
not yet succeeded with components is not a good candidate to suddenly succeed
with Web services. Only with measurable processes in place can the average
organization make decisions on this technology and then follow through to reach
a predictable, favorable goal.
Align, align, align
Gartner's Solon advised managers to
integrate key performance indicators into their performance management systems
to meet the goal of aligning application development efforts with business
And while Solon urged development leaders to contribute to corporate efforts
to measure the benefits of application development to the business, he cautioned
that such ''Return on Investment'' (ROI) questions can only be answered if the
business side is truly ready to try to measure business benefit.
Application development management must understand the drivers of
performance, said Solon, who groups the key input drivers of application
development into four categories: personnel, environment, technology and
processes (with software requirements counted as part of processes). The output
of all this is software that should comprise a mix of cost, efficiency, quality
and functionality goals. Measurement should be applied at various stages of the
process. Solon chides, however, that measurement without action is dangerous.
For one thing, when it becomes an ivory tower- or monastic-type discipline, it
starts to become less accurate, or inaccurate.
As corporate competitors embrace new technologies and gain advantage,
corporate management will be knocking on application development's door, seeking
to replicate or surpass those offerings. The message from Florida's AD Summit
was: ''Fine, but ... new technology is not the point.'' With proper practices in
place, taking on new technology will be less of a crazy adventure. Proper
practices here mean measuring -- taking time to take stock. CMM and function
points may help, but those are just some of the many varied means. The important
thing is to take the time to know the development organization as best as one
can, while still getting the work done to spec on time.
What do you think? Is measurement more trouble than it is worth? Is an
intuitive understanding of your team all that is needed? What about CMM or
function points? Voice your opinion. Send an e-mail to Editor-at-Large Jack
Vaughan at firstname.lastname@example.org.
Jack Vaughan is former Editor-at-Large at Application Development Trends magazine.