The Agile Architect

To Be Agile, Beware the Comforts of Ritual

Ritual is a comforting and seductive mistress. But how do you prevent it from holding you back? Our Agile Architect shares his thoughts.

• By Mark J. Balbes, Ph.D.
• March 6, 2013

I was at my usual spot in the war room the other day, working on the plan for the next release of our product. I had been through several days of ideation discussions at the white boards with my UI designer and usability expert. Together, we had figured out how the features would work, created wireframes of the user experience, written up the stories, reviewed and estimated them with the team, and were now ready to implement them. The last step was to write up the stories on 3" x 5" index cards and put them on the Kanban board. Carefully, I wrote out the cards, making sure the titles were big, neat and legible. I was smiling, content and extremely satisfied. This was the last step at the end of a long, exhausting creative process.

As I was doing this, one of my developers came over and asked why I didn't have an administrative assistant doing this. Leave it to an engineer to burst my bubble. Of course, he was right. There was no reason that I needed to be doing this work. It was completely rote and took a considerable amount of time; time I could have been spending on tasks that required my more specialized skills. But I liked doing it. It gave me a sense of satisfaction and closure. Could I really give up the pleasure of this practice to someone else? And for that matter, was there a way to change our process so that this rote tasks never had to be done? These were questions I didn't want to ask. I was greedy. I wanted to keep the task and the satisfaction I got from it for myself.

In this agile world, we call this kind of task a ritual. It's something that we do that gives us a false sense of accomplishment. It's easy because we don't have to think too hard about it. We "know" we have to do this task because our process tells us it's the next step and, when we are done, we get the satisfaction of checking it off the To Do list. We've stopped asking the question "Is this helping me deliver high quality software?" and instead the ritual has become an end unto itself.

Any task or process, big or small, can become a ritual. When we were doing eXtreme Programming, we eventually realized that pre-iteration planning and iteration planning had become rituals. We were doing them because the process said to do them. We learned about Kanban, which allowed us to jettison these rituals and the overhead associated with them.

When I help companies with their agile transformations, I see a lot of these rituals. Documentation that "Must Be Written" but no one knows why, meetings that "Must Occur" but never provide value and metrics that "Must Be Calculated" even though they are never used. Even worse, in large departments and companies, it may become someone's job to be the keeper of one or more rituals. In this case, the ritual is so embedded that there is an actual person who has a personal, vested interest in maintaining the ritual regardless of its value.

As you are reading this article, you're probably thinking to yourself, "Oh God, now he's going to start listing off a bunch of trite rituals that we should get rid of." And honestly, that was my original intention for this column. But let's face it. That's boring. I mean, anything you do, big or small can be a ritual, from how you update your bug tracking system to how you do release planning or subsets thereof. What's more important is how you think about the things you do and take the time to question everything.

So instead of a facile diatribe about rituals, let's talk about my favorite subject: Me. :-)

You see, while I've been writing this, my brain is still stuck on the first paragraph of this article. Not just the card writing part, but the whole process that I've just dedicated the last week of my life to. In my mind, I've been unwinding the process. My thoughts go something like this:

I'm spending a lot of time writing a stack of story cards. I know some of the cards can become out of date before we start implementing them, so why am I bothering to write them now?

I'm writing them now, because I already know what the stories are going to be, or at least I think I do. Why did I need to know so far in advance what the stories would be?

I need to know the stories so far in advance so that the team can estimate them. But why does the team need to estimate them?

My team needs to do estimates pretty far in advance of implementing the stories, sometimes as much as six months in advance, because our customer wants six month release cycles. He wants to be able to prioritize the work at the start of the six month cycle based on level of effort for each minimal marketable feature (MMF). We know the details of the MMFs will change but the overall effort for the MMF stays relatively constant.

Why does the customer want a six month release cycle? Because it's a military application. There are high costs associated with deploying a release that are not within our control, including the cost of training and field testing. It doesn't matter how good the software is, how stable or easy to learn. In the military, the standard creed is that you "Train as you fight," meaning that training must occur under realistic conditions. Deploying new software that can change how our users operate requires a large investment of their time.

So, unwinding this process, I've now identified the invariant. I can't change the six month release cycle or the reasons for it. I need to look at the next step that falls out from it: the need for estimates for release planning purposes, which leads me to the early creation of stories.

Is there a different way to get estimates that gives me as good or better estimates than actually creating the stories that will make up the MMFs that we plan to put into a release? Certainly I could just make up numbers for each MMF based on my professional experience. But the scientist in me doesn't like that. I like to be able to back up my numbers with some substantial facts. In the current process, we do this by having the team estimate the stories in each MMF and then roll those estimates up into an overall estimate for the MMF. Although we estimate in arbitrary XP units, we have a lot of data from past releases that allow me to turn 1 XP unit of effort into a realistic number of man-hours.

But while writing this article, with the tumblers of my brain turning while I think about my own rituals, I'm wondering if I could simply describe each MMF to the team at a high level, have them correlate the MMF with an MMF we've previously built, and then use those man hours as our level of effort. This isn't rocket science. In fact, it's a pretty standard way to do estimates; compare the future effort to something you've already done. But it's not how my team works now, and only by stepping back while writing this article have I given myself the opportunity to consider it. Thinking through the repercussions, it means that we won't have the satisfaction of having stories queued up before working on the release. However, as we've learned, this is a good thing. Our stories change anyway.

And with this change, the team may never have to estimate individual stories since we don't need them to satisfy our customer requirement to scope six month releases.

Now, at this point in my thinking process, I'm still skeptical. There are reasons we work the way we do now. So what do we lose? At first blush it strikes me that, without story-level estimates, we lose our ability to track our team velocity on a week-to-week basis. But there is an alternative. Before we start work on an MMF, we will still need to break it apart into stories. We know the level of effort we originally allocated to the MMF and we know the number of stories in the MMF. So, as we implement the MMF, we can easily track the percentage completion of the MMF based on the original estimate by tracking the percentage of total stories completed. In fact, we can track the percent completion of the release in the same way. And since we know the relative sizes of the MMFs in the release and how fast we are completing the current MMFs, we can extrapolate a completion date as well.

So there you have it. From wondering about my little ritual of writing up story cards to a massive overhaul of our release planning and estimation process, all in one article.

Will it work? I'll keep you posted. Just as soon as I finish writing up these cards.