Manager's View/What price quality?
- By Shawn Bohner
- June 5, 2001
Corporate it executives today face ever-increasing demands to show a
return on information technology. In cost-benefit analysis, however, the
cost of quality is seldom considered. But this is changing as the cost
of quality (CoQ) principles applied routinelyand effectively in manufacturing
emerge as important factors in the application value equation. The "good"
part of the good, fast, cheap equation is now getting its due along with
improvement programs such as GE Information Services' Six Sigma enterprise-wide
quality assurance program, total quality management (TQM) and the like.
from the Stamford, Conn.-based Meta Group shows that in 1998, more than
85% of IT shops conducted return on investment
(ROI) analyses on their IT expenditures (up from less than 50% in 1996).
Furthermore, with more than 30% of IT organizations involved in process
improvement initiatives (for example, the Software Engineering Institute's
Capability Maturity Model, and the Six Sigma and ISO 9000-based approaches),
many have faced or now face the challenge of showing ROI for improved
productivity and quality.
During 1999-2001, IT organizations will continue to be distracted
by year 2000 (Y2K), Euro, Webification and ERP initiatives. However,
by 2001-2002, software process improvement will be on the rise again
with a renewed interest in quality and agility measures. In the meantime,
ROI analyses will become increasingly commonplace, with companies showing
a growing interest in being able to express the cost of quality and
When the cost of quality principle was developed for the manufacturing
industry, the assumption was that it is always cheaper to do the job
right the first time. However, software challenges this assumption with
its requisite changeability. Software is supposed to respond to changes
in its functionality, quality or even its purpose -- otherwise its functionality
would be realized in the hardware. Software professionals and their
customers have therefore come to rely on software's ability to accommodate
changes, bringing new meaning to the phrase "doing the job right."
But before IT shops can justify investments in improving quality,
they must understand the costs of quality software. This can be done
by answering the following questions:
- How much does software failures caused by poor quality cost the organization?
- How good or bad is the software quality in the IT applications portfolio
(does it improve or devalue the asset)?
- How much does it cost to produce quality software?
- How much does it take to reach a financial break-even point (price/performance)?
To this end, the costs of quality fall into three primary categories:
the cost of non-conformance, the cost of evaluating conformance, and
the cost of preventing poor quality/ensuring good quality.
When dealing with non-conformance, both pre-release and post-delivery
quality issues must be addressed. Typical pre-release costs include
defect management, rework, re-reviews and retesting. For post-delivery
issues, technical support, defect notification, fixes and updates must
Evaluating conformance entails assessing the quality level of the
software produced in terms of identifying non-conformance situations,
as well as providing quality control checks. To determine the level
of non-conformance and to understand the condition of the software,
cost items such as inspections, testing, software quality assurance
and reviews will be encountered. For quality control checks, typical
costs include product quality audits and criteria for go/no-go decisions.
For many IT organizations, the costs associated with assessing quality
levels are frequently overlooked.
Effort must be expanded to set quality goals, establish standards
of performance and performance zones, and to develop and execute a quality
process improvement program that will develop quality products. Typical
costs associated with this effort include training, process improvement
initiatives, metrics collection and analysis, and the definition of
release criteria for acceptance testing.
Costs vs. benefits
For software quality, the end-game centers around controlling costs
and managing to the appropriate quality performance zones. Once the
three categories of quality costs -- non-conformance, evaluating conformance
and preventing poor quality/ensuring good quality -- are in place, several
benefits can be realized.
First, an organization's quality costs can and must be compared to
the benefits derived. Costs must be compared to the overall development,
delivery and, ultimately, maintenance and support costs. IT management
must understand that information is what the organization values and
that it is information technology that delivers this value. Therefore,
the benefit is demonstrated to the degree that the technology enables
the delivery of information that meets both the timeliness and quality
needs of the business. Masked costs associated with software quality
are now revealed to IT management. Economic trade-offs with quality
become apparent, which leads to more effective decision-making. The
result is that the effects of quality programs and process improvements
become measurable, visible and actionable.
Once IT executives see and understand the real cost of software quality,
the more thorough analysis will lead to proactive positions regarding
the competitive environment. For example, quality costs can be baselined
and benchmarked against industry norms. In addition, application development
and maintenance organizations can now deal with software quality in
an economic context (the bottom-line).
Most cost of quality techniques are applied retroactively to ROI,
as most IT executives want to know the payoff from their investments
in process improvement. However, there are other applications that IT
organizations should consider. Cost of quality can be used to:
- Determine the potential costs and risks of specific quality trade-offs on
critical software projects;
- Determine potential legal exposures associated with customer-experienced
defects (for example, Y2K exposure);
- Develop a basis for budgeting the quality management functions (control
- Couple software quality cost data with production cost data to demonstrate
the value of staff efforts to the corporate bottom-line; and to
- Compare proposed process improvements initiatives and select the most cost-effective
While executives struggle with IT business value and software post-Y2K
quality woes, the cost of quality is emerging as a key element in the
IT value equation. But be warned: Applying cost of quality principles
to apps means accounting for expenditures and understanding the economic
trade-offs of delivering quality software.
Dr. Shawn Bohner is senior program director of Meta Group's Performance Engineering and Measurement Strategies.