In-Depth

The Changing Rules of CM

• By Philip E. Courtney
• May 29, 2001

Corporate development organizations have heard many promises during the efforts to come up with a process for building high-quality software. Development methodologies, project and process management tools, modeling software and testing tools have all been part of the attempts to improve software quality.

A federal government-funded effort to build the Capability Maturity Model (CMM) for improving software quality also found a place in some of the world's largest development organizations. For years, many of these organizations touted their success by moving up the CMM ladder. Lately, however, IT organizations that once touted adherence to CMM, created by the Pittsburgh-based Software Engineering Institute (SEI), no longer speak of the model. And few new implementations can be found.

For example, during the research for several feature stories on quality and process for Application Development Trends, IT development managers did not mention CMM. Many IT managers talked about implementing development processes, but did not say whether CMM rules would be part of the process.

Such apparent changes in IT attitude could lead one to believe that CMM efforts have been displaced by large-scale conversion projects such as year 2000, the Euro or application integration. It can also cause one to consider that perhaps the message has reached critical mass and the organizations that needed to buy into the model have already done so. Other schools of thought place CMM into the category of "too-difficult-to-achieve," while still others just have not gotten the word.

Quality experts say the buzz quieted mostly because CMM supporters no longer view the model as doctrine that requires an all-or-nothing implementation. Instead, companies are successfully inserting pieces of the CMM model into proprietary processes that better fit their industry and/or company.

"It [CMM] is still out there and stronger than ever," maintains Karen McKeown, manager of corporate SEI compliance at Keane Inc., a Cambridge, Mass., consulting firm. "It seems, though, that through the years, a lot of companies tried implementing the [undivided] model on their own and failed."

Observers could ask how full implementations of CMM could possibly fail. The Capability Maturity Model was created primarily to deter failure by improving processes and quality.

"The failures can typically be attributed to taking CMM as gospel," explained Wolfgang Strigel, president of the Software Productivity Centre, Vancouver, B.C. "CMM is very rigid. We have always suggested using it as a framework for industry best practices." Strigel suggests that organizations combine CMM pieces with standards unique to their industry and company.

While CMM purists may cringe at the suggestion, extending the model does not run contrary to SEI philosophies, say officials there. "Taking the best practices is not necessarily against the model," said Bill Pederson, manager of the Software Engineering Process Management Program at SEI. "We recommend you take a look at your business needs and use the model to meet your business objectives."

Still, despite the recognized need for improved quality at most companies, word of the Capability Maturity Model has not reached everyone, everywhere. According to Randy Rice, president of Rice Consulting Services Inc., Oklahoma City, the need to provide a CMM overview invariably arises during the classroom training his company delivers regarding the implementation of quality testing practices. "We frequently need to explain some of the high-level concepts to IT groups," noted Rice. "We'll also show how specific requirements management and testing disciplines link to the different levels of CMM."

SEI and CMM in brief

Established in 1984 by the U.S. Department of Defense (DoD), the SEI is a federally funded research and development center. An integral component of Carnegie Mellon University, and sponsored by the Office of the Undersecretary of Defense for Acquisition and Technology, the SEI has a broad charter to address the transition of software engineering technology. Its mission is carried out through two principal areas of work, Software Engineering Management Practices and Software Engineering Technical Practices. The latter practice aims to help software engineers improve their ability to analyze, predict and control selected functional and non-functional properties of software systems. The management area contains the CMM practices, and concentrates on organizational ability to predict and control quality, schedule, cost, cycle time and productivity when acquiring, building or enhancing software systems.

Although CMM has traditionally been used in the singular sense, there actually exist several different models, each developed and expanded to meet the diverse needs of differing IT organizations. For example, while the SW-CMM is the Capability Maturity Model for Software, the SA-CMM represents the Software Acquisition Capability Maturity Model. There is also a model for Systems Engineering, Integrated Product Development and People.

One of the newest SEI projects is the Capability Maturity Model Integration (CMMI), which is said to address software and services. The CMMI plan calls for producing a product suite that can provide industry and government with a set of integrated products that support process and product improvement.

"There are a lot of different requirements in the industry," explained Mike Phillips, the SEI's CMMI project manager. "In some cases, for example, IT shops may not be into development as much as others and instead are into purchasing commercial off-the-shelf software. Others are working with service organizations to integrate systems. These are organizations that are looking for elements of CMM to improve processes."

Strong government ties

Forged from DoD requirements, CMM has strong ties to the government. So strong, in fact, that a new DoD directive issued in late October 1999 calls for full Level 3 compliance for each contractor performing software development or upgrades for use in any Acquisition Category 1 (ACAT I) program (ACAT 1 programs are "Major Defense Acquisition Programs"). [See "CMM: Five levels of maturity," this page.]

But CMM disciplines have spread well beyond government agencies and government customers to include a large number of industries. "Although CMM grew out of government requirements and encompasses government sub-contractors and suppliers, we've seen an enormous amount of activity in the commercial sector," said McKeown at Keane.

"We cannot afford to ignore it," said Bruce Kantelis, vice president of technology at LifeCare Technologies, Clearwater, Fla. "We looked to CMM for help in improving our processes and our quality. The model is viewed as a way to help us stay competitive."

LifeCare Technologies is currently implementing the disciplines required for the company to reach Level 2 certification. CMM did not gain the attention of development managers until LifeCare began looking for ways to improve its processes. "We latched onto CMM as part of an overall quality improvement campaign because it helps establish baselines of behavior," explained Kantelis.

One sought-after baseline behavior -that is also a large component of Level 2 - is a solid process for requirements definition and management. After an internal evaluation revealed methods for process improvement, LifeCare Technologies implemented the Caliber-RM package, from Atlanta-based Technology Builders, for requirements management. Product managers for the various technologies produced by LifeCare used the Caliber-RM tool to establish product requirements as well as the use cases, which are definitions of how the technology would be employed at customer locations.

"The product manager has to prove the use case through the requirement and vice versa," explained Kantelis. "All tasks are performed using a team approach, and everything adheres to the processes defined in the model."

Still, according to Rick Miller, project manager for Renaissance Government Solutions, a division of Renaissance Worldwide based in Newton, Mass., impact analysis tools can be crucial components within any large-scale process for accurately analyzing requirements. "A tool that easily provides answers to 'what if' questions as they relate to application changes helps provide a better picture for costing and project timelines," he said.

Using the ASG-Vista tool from Allen Systems Group, Naples, Fla., Miller's Renaissance consultants have reverse engineered legacy code to create object-level representations of an application portfolio. The object-level representation allows for quick viewing of an application flow from a procedural level, as well as from the decision points within each individual program. "Not only does this reduce an application learning curve from weeks or months to hours or days," it allows users to easily view the impact of proposed changes and accurately establish a project plan, said Miller.

Where has CMM gone?

Despite the many benefits made possible through the implementation of quality-driven procedures, why did the Capability Maturity Model remain unmentioned during so many past conversations?

"Because it's going under a different name," theorized Keane's McKeown. "The large-scale CMM efforts that failed in the past are taking CMM basic disciplines today but calling them a different name."

Another reason, contends Uttam Narsu, senior analyst at Giga Information Group, Cambridge, Mass., is that CMM is perceived as a heavyweight, long-term directive that tends to frighten commercial IT organizations. "It's difficult to quantify the return on investment, it takes a lot of effort to get evaluated, there's little if any marketing value and there's controversy on how to successfully move between levels," he said. "It's also possible that IT organizations don't want to spend the time and resources to implement the specific CMM processes."

Rice at Rice Consulting Services tends to agree: "The general reaction I see is that people feel it all makes sense, but they become discouraged when they see that only a handful of companies have made it to Level 5."

Furthermore, advancing to new CMM levels and actually certifying any advancement has oftentimes proven to be a difficult process, as the SEI has to-date certified only a limited number of qualified assessors. SEI's Pederson said that could simply be a matter of perception, but contends it is more likely a bootstrapping issue. "We currently have approximately 300 lead assessors, so the market is not that constrained," he said. "However, we won't accept people onto the assessment team unless they've gone through two assessments. The problem is that by the time an assessment opportunity arises, it may be with a company that competes with the company for which the candidate-assessor is employed. Chances are they'll have to wait for the next assessment opportunity."

Regardless of their feelings about CMM, both Rice and Giga's Narsu are firm believers in the importance of process. "Many people see the exceptions within CMM and apply the best of breed to their own processes," said Rice.

"It's important to drive change to meet business objectives," noted Narsu. "Don't look at the CMM as a wholesale solution; use it as a good source for key adaptations."

And that is just what an increasingly larger number of firms are doing, said Strigel at the Software Productivity Centre. With his company focused on core process improvement and providing training for project/process management, Strigel has witnessed users shift away from treating CMM as a religion to taking the portions that make the most sense for improving quality. "The highest return comes from implementing a requirements model and a process improvement model," he said. Whatever it may be called, CMM and its disciplines appear to be alive and well - in some form - as organizations continue to embrace at least the concept of quality processes and process improvement.

"The data indicating improvement speaks for itself for those companies that have implemented the model," said SEI's Pederson. "However, it's important to note that installing CMM and getting the disciplines are important, but installing CMM and creating a bureaucracy is wrong."