The Agile Architect
Agile and CMMI, Part II: Tips from the Trenches
Facing CMMI appraisal? Mark walks you through how he completed the Risk Management and Decision Analysis portions of CMMI Maturity Level 3 for his agile shop.
- By Mark J. Balbes, Ph.D.
- October 20, 2011
So your boss comes to you and says, "Guess what! We're getting CMMI 3 certified and you get to do the prep work." After several dark thoughts of revenge, vain attempts to deflect the work to someone else and a few minutes of self pity, you resign yourself to the fact that you are going to have to at least live through this arduous task and, at best, try to make something of it.
If the above happens to you (as it has to me), fear not! You work in an agile shop, which means your company is already following most (if not all) of the CMMI Maturity Level 3 practices. Therefore, the question isn't, "Are we CMMI compliant?" No, the question is how to prove it!
In the space I have available in this column, I can't give you an exhaustive review of evidence for every process area. What I can tell you about is our experience: That we had a relatively easy time with the process areas that dealt with the projects themselves -- where we needed to do work was around organizational practices and evidence.
First the easy part:
Projects and Process
From a project point of view, evidence is generated by standard agile practices. While your team may try to avoid long meetings with useless meeting minutes, your activities will generate output that acts as evidence. Pre-iteration planning and iteration planning meetings generate stories, acceptance criteria, story prioritization, and buy-in from the customer and development team to the plan. Even without formal meeting minutes, your teams will have some mechanism of documenting these decisions for reference during the iteration. If you are using a Kanban board, the board itself is documentation of this plus it documents your development process. The appraisers can be shown a photo of the Kanban board or you can show them the board itself.
Automated tests act as documentation for the software, retrospectives write-ups are evidence of many things, most especially process improvement initiatives.
What We Were Missing
In my
last column, I flat-out said that we were after a CMMI appraisal for the money -- that we needed it to help win government contracts. While that is true (albeit somewhat exaggerated for dramatic effect), the reality is that going through the steps to prepare for the appraisal has helped us to become a better company. Agile development has such an in-the-moment feel to it that sometimes important aspects of a mature development environment are left out.
What we found was that our organization was lacking in two process areas:
- Risk management
- Decision analysis and resolution
This isn't to say that we didn't do these things -- we did. (And we're pretty good at it.) But we had no mechanism at the company level to help us. In other words, we did them well because we have good people who know how to do their jobs. But with every team inventing for themselves their processes and procedures, this leads to duplication of effort and waste.
Let's take these process areas one at a time.
Risk Management
As I was preparing for our appraisal, risk management was the hardest process area for me to get a handle on. After all, agile processes are all about minimizing risk. We do sprints/iterations to minimize the risk of building the wrong thing. We write automated tests and pair program to minimize the risk of delivering low-quality software and/or delivering it late. We involve the customer to minimize the risk of delivering the wrong functionality or delivering the right functionality but expressed in an unintuitive or incorrect way.
Despite this, when we looked at the evidence we had, we were unable to show how we mitigate specific risks to our individual projects outside of the kinds of general risks that agile addresses. As examples, agile development practices don't give specific risk mitigation strategies to address a dependency on a third party vendor, a disinterested customer or an understaffed team. We had no stated mechanism to evaluate risks to decide which needed to be monitored and/or mitigated, nor what criteria would be used for that evaluation -- all things required by CMMI.
Enter the project charter.
In preparation for our Maturity Level 2 appraisal last year, we had introduced the idea of a project charter. The charter was the mechanism we used to ensure that important issues were thought about when starting a project, and that these same issues would be revisited on a regular basis. The company maintains a charter template that has six sections comprised of 21 questions that we've decided are important for any project; questions like "Who is your customer" and "What type of cycle planning will the team use?" Part of our retrospective process is to regularly review the charter. In fact, one of the questions in the charter is "When will the team review the charter?" The challenge with creating and maintaining the charter is keeping it complete enough that it is useful and short enough that the team isn't overly burdened to maintain it.
Although the CMMI Risk Management process area has seven specific practices, I only added four questions to the project charter template:
- What are the sources of risk to this project?
- What are the specific known risks to this project?
- How will you monitor and track these specific risks?
- How will you mitigate these specific risks?
The critical issue for the team is to determine which risks need to be listed in their charter. Part of CMMI risk management is to prove that you have a mechanism to define parameters that will be used to analyze and categorize risks. These will be different for different companies. If you were a spreadsheet person, you might be tempted to require quantitative metrics to measure the risk; things like frequency, user impact and financial impact. While useful in some situations, trying to come up with a set of key quantitative metrics didn't feel like the right approach for us. Instead, I asked myself why we were really interested in risks from a company perspective. The instructions I wrote for question three in the charter template were:
Risks listed here should meet one or more of the following criteria:
- Possible schedule slip that is more than allowed by the contract or schedule.
- Possible cost overrun paid by us by more than agreed to with our upper management.
- Possible cost overrun paid by the customer by more than agreed to by the customer
- Possible cost underrun which represents loss of profit to us
- Risk to delivery of contractually obligated deliverables. Examples are:
- Technical risk associated with delivering a new feature
- Dependencies on 3rd party software e.g. that are not yet released or well tested
Risks that do not need to be listed are those that are mitigated by our standard agile development processes. For example, the risk of delivering buggy software is mitigated by use of TDD and QA testing. Risk of delivering the wrong products is mitigated by regular demos and iteration planning with the customer.
Other risks, such as lost work or time due to natural disasters are covered at an organizational level and do not have to be addressed by individual projects.
The charter template itself -- along with the project charters that answer the questions posed in the template -- give us the evidence we need to prove that we handle risk management for risks not already addressed by our agile practices.
Decision Analysis and Resolution
Decision analysis and resolution is closely related to risk management. After all, your critical decisions are probably related to your critical risks.
Agile software development is all about making decisions with the most information possible. Contrary to traditional methods that try to minimize risk by making decisions early, agile methodologies want to delay making decisions until the last responsible moment. This is the moment when delaying making a decision will cause more harm than making a decision.
While we have many good developers in our company, we have very few traditional project managers. For the most part, they just don't fit with our culture. However, this means we have very few people trained in any formal decision-making processes.
Rather than trying to push a specific decision-making methodology on each team and train them on how to do this, we instead set up a decision facilitation team made up of experienced decision makers. We then put in place a mechanism for the teams to request a decision facilitator. In this way, we don't have a high overhead to put these practices in place.
At this point, we did have to generate our evidence of using the decision facilitators. So we asked our teams being appraised to choose a question they needed answered and go through this process. Honestly, we did this purely to generate evidence for our appraisal. The teams weren't exactly pleased with us and they groaned in agony when trying to come up with a question.
But then a funny thing happened.
All the teams came up with questions that seemed relatively benign and yet, out of this process came sweeping decisions that changed the way the teams worked. Perhaps the best example of this is the team that asked the question "Do we keep two sub-teams or merge them into one?" Out of the decision making process, they realized that the issue wasn't whether to have one team or two. Instead, they realized that their project manager was tasking and prioritizing stories for the sub-teams completely independently. This was causing conflicts between the two teams since they had real dependencies on one another. Out of the decision process, they abandoned the original issue as a non-issue and instead merged their Kanban processes into a single process. This forces the teams and project manager to address dependency issues as part of their everyday work.
Was It Worth It?
While the CMMI appraisal was a pain to deal with and I honestly would have preferred spending my time developing new software, the benefits far outweigh the drawbacks. Our company has emerged stronger and there are more organizational resources for teams to draw on when they are most challenged.
And so, for now, we leave the rather dry topic of CMMI, and on to a preview of my next Agile Architect column:
Now just imagine…It's a dark and stormy day. In the conference room, you've just finishing your pitch to upper management about why you should move your company from your current development practices to an agile approach. You've laid everything out like a lawyer making his case to the jury. They laughed in the right places. They nodded in approval. And as you wrap up your masterpiece presentation, the thunder outside cracks and the president of the company chimes in. "That's nice in theory but it'll never work in practice."
What do you do now? Stay tuned!