Columns

I/T Value: Behind the numbers

• By Tony Baer
• September 1, 2004

A little more than a year ago in my column “Three little words ” (Application Development Trends, July 2003, p. 25), I recounted my impressions from a forum on the return on investment (ROI) for business intelligence (BI) projects. At first blush, the notion of documenting returns on business intelligence projects should be a no-brainer because these applications are very metrics-oriented, and are intended to help companies understand where their margins or operations could bear improvement. So if you run a sales and marketing data warehouse that allows you to identify whom your most profitable customers, products or regions are, it can provide the data you need to help you modify your sales and marketing strategies to generate better returns.

Yet when we polled a room full of business intelligence professionals at a conference panel session on the topic, we found relatively few hands going up when we asked whether they routinely used return on investment measures to justify the cost of their projects. Instead, we were lectured about the importance of “soft” benefits, such as process improvements that yield new business opportunities.

What a difference a year makes.

We addressed the same forum this year, posed the same question, and found roughly one-third of the room indicating that return on investment was now standard procedure for getting new business intelligence projects green-lighted.

“It used to be that if you had a [sound] business case, it would get approved. Now things go over to a committee,” explained one panelist on the change in climate.

Of course, maybe the change was due to the makeup of the audience -- this year’s session drew a much higher percentage of representatives from the business side of the organization. Or maybe the long-term impacts of information technology budgets were finally sinking in on application development or line-of-business organizations.

Nonetheless, when we asked about the newfound attention to return on investment, we drew an interesting assortment of responses. These ranged from the need for making project teams accountable, to performing the financial due diligence required in publicly traded corporations, to arming projects with “objective” numbers to compete against rival proposals. And some responses involved something even less tangible: knowing the value of information.

But what is the value of information? According to one respondent, it is inversely related to the costs of not having the data. He maintained that the value of data -- such as trends in the profitability of a product line, sales region or customer segment, or the efficiency of a business process -- becomes obvious if the information is actionable. For that reason, he maintained that the return on investment of business intelligence projects is actually less tangible than, say, a financial accounting application.

Few in the room would admit to relying entirely on soft benefits for justifying project proposals. But the sense was that the intangibles could act as effective reality checks on any proposal because there is a need to validate what returns are being tracked.

For instance, a reporting system might save labor in producing reports, but if your organization’s goal were to increase the top line (sales), such a figure would prove irrelevant. Or an analytic system designed to dissect operational problems in one area could return a positive return on investment if the system correctly identifies the bottleneck. But if the net result finds the resolved bottleneck being moved to another operation or part of the organization, you could find yourself in a scenario redolent of “The surgery was successful, but the patient died.”

Similarly, just as hard return on investment numbers can be taken out of context, so too can soft ones.

“For our business intelligence application, we didn’t even figure the intangibles because they were so large,” one panelist said.

Therefore, while there was a consensus that return on investment numbers are becoming more important for business intelligence projects, just any numbers won’t do. And even if you come up with what appear to be watertight metrics, you may still have to be wary of the business intelligence equivalent of the Heisenberg uncertainty principle, which states that the act of measuring can impact the results. For instance, when employees know that their performance is being measured, they will temporarily work by the book. You’ll gain a 5% boost in productivity, but it will wear off once workers return to their old habits.

The moral of the story? When tracking people, you can’t simply impose metrics from above. Get their input in designing metrics and your return on investment will be far more bulletproof.