Columns

Manager's View/What price quality?

• By Shawn Bohner
• June 5, 2001

Corporate it executives today face ever-increasing demands to show a return on information technology. In cost-benefit analysis, however, the cost of quality is seldom considered. But this is changing as the cost of quality (CoQ) principles applied routinelyand effectively in manufacturing emerge as important factors in the application value equation. The "good" part of the good, fast, cheap equation is now getting its due along with improvement programs such as GE Information Services' Six Sigma enterprise-wide quality assurance program, total quality management (TQM) and the like.

(ROI) analyses on their IT expenditures (up from less than 50% in 1996). Furthermore, with more than 30% of IT organizations involved in process improvement initiatives (for example, the Software Engineering Institute's Capability Maturity Model, and the Six Sigma and ISO 9000-based approaches), many have faced or now face the challenge of showing ROI for improved productivity and quality.

During 1999-2001, IT organizations will continue to be distracted by year 2000 (Y2K), Euro, Webification and ERP initiatives. However, by 2001-2002, software process improvement will be on the rise again with a renewed interest in quality and agility measures. In the meantime, ROI analyses will become increasingly commonplace, with companies showing a growing interest in being able to express the cost of quality and agility.

Justifying quality

When the cost of quality principle was developed for the manufacturing industry, the assumption was that it is always cheaper to do the job right the first time. However, software challenges this assumption with its requisite changeability. Software is supposed to respond to changes in its functionality, quality or even its purpose -- otherwise its functionality would be realized in the hardware. Software professionals and their customers have therefore come to rely on software's ability to accommodate changes, bringing new meaning to the phrase "doing the job right."

But before IT shops can justify investments in improving quality, they must understand the costs of quality software. This can be done by answering the following questions:

• How much does software failures caused by poor quality cost the organization?
  
  
• How good or bad is the software quality in the IT applications portfolio (does it improve or devalue the asset)?
  
  
• How much does it cost to produce quality software?
  
  
• How much does it take to reach a financial break-even point (price/performance)?

To this end, the costs of quality fall into three primary categories: the cost of non-conformance, the cost of evaluating conformance, and the cost of preventing poor quality/ensuring good quality.

When dealing with non-conformance, both pre-release and post-delivery quality issues must be addressed. Typical pre-release costs include defect management, rework, re-reviews and retesting. For post-delivery issues, technical support, defect notification, fixes and updates must be considered.

Evaluating conformance entails assessing the quality level of the software produced in terms of identifying non-conformance situations, as well as providing quality control checks. To determine the level of non-conformance and to understand the condition of the software, cost items such as inspections, testing, software quality assurance and reviews will be encountered. For quality control checks, typical costs include product quality audits and criteria for go/no-go decisions. For many IT organizations, the costs associated with assessing quality levels are frequently overlooked.

Effort must be expanded to set quality goals, establish standards of performance and performance zones, and to develop and execute a quality process improvement program that will develop quality products. Typical costs associated with this effort include training, process improvement initiatives, metrics collection and analysis, and the definition of release criteria for acceptance testing.

Costs vs. benefits

For software quality, the end-game centers around controlling costs and managing to the appropriate quality performance zones. Once the three categories of quality costs -- non-conformance, evaluating conformance and preventing poor quality/ensuring good quality -- are in place, several benefits can be realized.

First, an organization's quality costs can and must be compared to the benefits derived. Costs must be compared to the overall development, delivery and, ultimately, maintenance and support costs. IT management must understand that information is what the organization values and that it is information technology that delivers this value. Therefore, the benefit is demonstrated to the degree that the technology enables the delivery of information that meets both the timeliness and quality needs of the business. Masked costs associated with software quality are now revealed to IT management. Economic trade-offs with quality become apparent, which leads to more effective decision-making. The result is that the effects of quality programs and process improvements become measurable, visible and actionable.

Once IT executives see and understand the real cost of software quality, the more thorough analysis will lead to proactive positions regarding the competitive environment. For example, quality costs can be baselined and benchmarked against industry norms. In addition, application development and maintenance organizations can now deal with software quality in an economic context (the bottom-line).

Most cost of quality techniques are applied retroactively to ROI, as most IT executives want to know the payoff from their investments in process improvement. However, there are other applications that IT organizations should consider. Cost of quality can be used to:

• Determine the potential costs and risks of specific quality trade-offs on critical software projects;
  
  
• Determine potential legal exposures associated with customer-experienced defects (for example, Y2K exposure);
  
  
• Develop a basis for budgeting the quality management functions (control and assurance);
  
  
• Couple software quality cost data with production cost data to demonstrate the value of staff efforts to the corporate bottom-line; and to
  
  
• Compare proposed process improvements initiatives and select the most cost-effective ones.

While executives struggle with IT business value and software post-Y2K quality woes, the cost of quality is emerging as a key element in the IT value equation. But be warned: Applying cost of quality principles to apps means accounting for expenditures and understanding the economic trade-offs of delivering quality software.