The Agile Architect

What Does an Architect Do in an Agile Shop? An Agile Architect Explains All...

If you're wondering what a software architect does in an agile development environment, you're not alone. Mark J. Balbes shares what he does and how it all comes together in better code.

• By Mark J. Balbes, Ph.D.
• August 17, 2011

Almost four years ago, I started my new job as the first and only vice president of architecture at Asynchrony Solutions. I sent an e-mail to all employees introducing myself as such, and I got the following response from one of the developers:

"So what does an Architect do in an architecture-antagonistic TDD/Agile/XP shop?"

My response was:

"The title is almost meaningless. :-) My aim is to have successful agile projects."

Honestly, I hadn't thought much of my title when I got it. I've had a lot of titles in the past, from "software engineer" to "system-of-systems software architect." My new title was just a title for a job that had very little definition.

I did know going in I wasn't going to be spending my time writing architecture documents. Agile principles are in direct conflict with a traditional software architect role -- in an agile project, where the creation of working, tested code trumps everything else, nothing separates architecture from design and implementation. They are all wrapped up together in the practices we use to produce the code.  Instead of traditional documents and diagrams, the architecture is going to be exposed through the software itself.  

So what does a software architect do in a truly agile company?

Architecture Is a Frame of Mind
I've concluded that in an agile environment, architecture is a frame of mind. There is no room for an "ivory tower" architect that sits at a desk creating diagrams and making architectural rules for development teams to follow. Agile development favors person-to-person communication over written documentation. If you're an agile architect, the best place to be is in the war room with the development team, working alongside them every day. This provides a deep level of understanding about the problems being solved by the team that cannot be learned through traditional means.

The architect's role, then, is not to dictate to the team, but rather to guide, mentor and cajole the team in the right direction. Perhaps just as important, it is to help the team regroup after heading down a path that turned out to be less than optimal. Even a spectacular failure, large or small, gives the team and the architect a chance to learn and emerge with better software.

A common occurrence with the team I work most closely with is to create a solution that works, realize that there is something better we can do, and then evaluate whether we want to implement the new solution immediately or wait until it is needed. The agile approach says that refactoring should be done now and adding new functionality should be delayed until it is needed. Often these forces are at odds with each other. Are we refactoring or adding a new feature? While the developers are pondering this from an implementation and low-level design standpoint, I'm looking at the problem from an architectural point of view. Given the new solution, what is the leverage we gain as we start to implement the next set of stories? If the new solution is moving the architecture of the product in the right direction, I will often push for it even if the team doesn't see the immediate benefits.

So why me? What gives the architect the insights that the other members of the team don't see? Besides the obvious (experience, technical knowledge), the reality is that the architect operates in a different world from the rest of the team.  Whereas they are largely consumed with the day-to-day tasks of writing and testing the software, I am spending time at all levels of the project, from working with customers, to helping to write stories, to actual coding and testing. In addition, I interact with our company's upper management and help with other projects. Perhaps my biggest advantage is that, while I do spend some time coding, when I think about our software I am not constrained by the fear that I will have to implement my own ideas. It may seem crass, but as someone who has worked long years in the trenches I know that there is a subconscious tendency to discount ideas that may have huge long-term benefits if it means a lot of work in the short term.

The War Room Is the Playground of the Agile Architect
As mentioned earlier, the war room is where the real work of the agile architect gets done. The hustle and bustle of development is exciting and provides almost everything I need to do my job. In the war room, I can listen to what is going on with the team, shepherd the team to keep the software development moving in the right direction and mentor team members on the concepts they may not be familiar with.

But for all of this, I am by no means the first and last word on our project. Instead, it's more like riding a wave and trying to steer in the right direction. On a well-oiled team, the developers make the wave. They provide the technical expertise and know-how to build the software. This creates a tremendous momentum that moves the software forward whether I'm there or not.

What I can do is influence using a variety of skills through various means.

The Many Roles of an Agile Architect
I've found that as an agile architect, listening is my most important skill. In the war room, I have positioned myself so I can hear all that goes on. (Yes, I have an office somewhere. If I could only remember where it is...). I can eavesdrop on the conversations between the programming pairs and keep tabs on all the decisions that are constantly being made. This is not to micromanage, but rather to allow me to maintain my mental picture of the state of the software. It also allows me (as well as other team members) to interject in the conversation when needed. This might be to ask a clarifying question, inject some new information or tie the conversation back to higher-level ideas or constructs we've built or plan to build.

On our team, we have constant, ad-hoc team discussions about how to implement a story or feature.  There may be disagreements among team members on how to implement the story. Part of my job is to listen to the discussion, pick out the important aspects of each side, help the team see the advantages and disadvantages of each proposed solution then come to a consensus.

While the team is often in the weeds, I am thinking about how to tie the story into the higher level architecture of the software and, perhaps more importantly, how we implement just enough of that architectural vision to get the current story completed while moving the software toward that vision

I also act as a mentor to the team. While I am certainly not the strongest technical member of the team, I can often see potential solutions that the rest of the team misses because I am looking at the problem from a different perspective. When a new member joins the team, it is up to me to teach them the higher-level concepts behind the software. Even for our experienced technical members, I mentor them on how to approach a difficult problem and what qualities we are looking for in the solution.

I am a shepherd for the team. There's a saying, "The road to hell is paved with good intentions." That is definitely true for agile software development, where a series of small solutions can snowball into an unwieldy mess. The problem arises out of the agile philosophy of doing the simplest thing possible to solve a problem and not trying to solve tomorrow's problems today. This philosophy works well most of the time. However, there are occasions when a series of simple enhancements builds to an unworkable or unmaintainable solution. It's my job to see when that is happening and change course before we waste too much time and effort.

Putting It All Together

Here's a concrete example of just one of many times when all of these skills have come into play:

One of our applications, The Mobile Field Kit, provides communication and data sharing capabilities to emergency first responders using tablet computers. It contains a set of tools (chat, shared white board, shared map and many more) that allows each first responder to see what the others are doing and share data seamlessly over a wireless mesh network. A large part of our technical challenge is sharing data between computers. This is needed both to send live data and to synchronize missed data when a computer drops off the network and later rejoins it.

Originally, we had two mechanisms for the two cases. Live data was shared as it was created using small, non-persistent messages. Each tool consumed these messages, updating their current state both in memory and in the local database. State-based synchronization was used when a computer went offline for any period of time. That is, if a computer lost network connectivity, when it reconnected, each tool would synchronize its current state across computers. We did have some shared, low-level data transmission capabilities, but each tool had its own tool-level synchronization mechanism. And each tool synchronization implementation was slightly different depending on which developers implemented it.

From an agile perspective, this was a decent design. The developers were happy with it. We had reusable code at the low level, and we knew how to turn the crank to implement synchronization for any new tool. However, I wasn't satisfied with this. We had reached the limits of reusability. I didn't like the fact that we had to take the time to build synchronization into every tool. In addition, it was a source of inconsistent behavior between tools, created hard-to-find bugs and was impossible to optimize without touching every tool. I knew that for our application to be able to scale to many tools and to continue to improve performance we had to have a single synchronization mechanism that each tool got for free. That is, for a new tool there would be no additional coding needed for data synchronization, period. I wanted to use the mechanism we already had for live data. Rather than state-based synchronization (the tools ask for what their current state should be), I wanted the system to simply catch them up on all the small live data messages they had missed. In other words, there would be no difference in behavior of the system when it was receiving live data versus when it was synchronizing missed data.

The team thought I was crazy. The solution wouldn't scale, they said. It would be slow and inefficient. I listened to all of their concerns and addressed them. I explained why our current mechanism wasn't optimal and all of the benefits that the new solution would provide. (And there are many more that are outside the scope of this article.) And, in the end, using all of my persuasive skills coupled with my vast experience and airtight arguments (sigh), I still couldn't convince them.

So I asked them to give me two weeks to prove out my ideas; if, at the end of two weeks, we didn't like where we were heading, we could always abandon the work. One of my best developers (let's call him Ben) was adamantly against the idea. Ben knew that there would be performance and scalability issues that didn't exist in the current system. Of course he was right, but I had confidence we could overcome them. Also, by overcoming them, we would make the entire system better, not just the individual tool. Since it was obvious that Ben understood what I wanted, even though he didn't agree with it, I asked him to take point on the initiative.

At the end of the two weeks, a remarkable thing happened. We were discussing where we were and how to move forward. Ben started with "Now I know I've been championing this idea..." He and the rest of the team had completely forgotten that I was involved. They had taken my ideas, made them their own and took complete ownership. I wish I could say I was clever enough to have foreseen this, but I kept quiet while the team decided to continue development of the new synchronization framework.

Three years later, we are still going strong. While I never sat down and coded out the solution, I was able to guide the solution to where it is now by being in the war room, continuing to monitor progress. We built up the framework in an agile fashion. As new stories were implemented, I made sure we took advantage of them to make our data synchronization framework stronger and faster. We've built other capabilities on top of it (like a rewind and fast-forward capability) that we could never have accomplished with our old architecture. And all I did was ask the team to humor me with a crazy idea and then nurture it over time. They did the hard part -- they implemented it and made it work.

It's All About the Code...
The agile architect is a jack-of-all-trades, doing what is necessary to ensure architectural integrity of the product.  Listening, guiding, mentoring and cajoling are all part of the job.  While the team owns the solution to a problem, it is the architect's job to state what the qualities of the solution should be.  And when something is particularly tricky or important, the agile architect rolls up his sleeves and gets to work coding with the rest of the team.

Regardless of any ideas, visions or documents, the code is the ground truth and the team is the keeper of the code. The agile architect must understand the current state of the software and know where it is going. He must spend his time with the team and be involved in all aspects of the project. The days of the ivory tower architect, sitting in an office producing diagrams and documents that may or may not be relevant, are dead. Long live the agile architect!

Questions or comments for Mark? How does your experience with agile architecture compare? Post in the comments and let us know!