News

Best Practices: Making a Microsoft Build

• By Mike Gunderloy
• July 24, 2006

"The question is," said Humpty Dumpty, "which is to be master—that's all." — Lewis Carroll, Through the Looking Glass

If you've ever had the chance to supervise the build lab for a major software project, with hundreds of computers chugging along and all doing your bidding, then perhaps you've felt something of Humpty Dumpty's satisfaction with his command of the English language. If, on the other hand, you still aspire to that lofty position, then this book, Vincent Maraia's The Build Master (Addison-Wesley, 2006), may be just what you need to get started. Based on the author’s experiences, the cover boldly promises to deliver "Microsoft's software configuration management best practices." Depending on your own needs, this could be a winning prescription.

Who Needs a Build Master?
Maraia says that the book applies to development shops of all sizes, but then he immediately goes on to qualify that statement to start it off with "small groups" of 40 to 100 developers. I don't know about you, but I've spent a good chunk of my career in situations where a small group of developers was three to five people. Sometimes it's even been me all by myself. Below a certain critical mass, you almost certainly don't need a build master. That is, you don't need a full-time employee whose entire job it is to make sure that the software you're creating can be built reliably and repeatedly on a regular schedule.

But that doesn't mean you won't benefit from this book. First off, there's always the chance that your small team will grow into a large team, and if you plan from the start to support a sane build process you'll be in a much better position than if you try to impose sanity on top of chaos later on. But beyond that, even if you don't require a dedicated build master, you still have the essential issue of turning source code into finished product. Whether you're doing this for five minutes a day or eight hours, the tips and ideas here can help you get your process under control. So think of this as a book, not just for the person with "Build Master" on his business cards, but for the build master in all of us.

Getting on the Mainline
The author's first major point (and the foundation on which much of the rest of the book depends) is that you have to get your source code repository configuration correct. If you don't have your tree configured correctly, your build process will be a mess no matter how good your tools and your team are. Maraia's recommendation is simple:

There should be a single source tree that is owned by the Central Build Team and, if needed, that could be synced up, built, and deployed on any day of the year.

This "golden master" or mainline tree is the heart of a successful build process. To some extent this seems obvious: If you don't know which set of bits are supposed to be going into the box, how can you have any hope of shipping the product? But I've seen some surprising things in my time, including utter train wrecks where different developers had the master bits for different libraries on their own workstations and creating a build involved collecting things from here and there and (hopefully) integrating them together.

Maraia goes on to explain how Microsoft combines this notion of a golden master tree with Virtual Build Labs (VBLs) to enable massively parallel development on large projects. By having individual teams work in their own copies of the master tree, with their own build process, and enforcing strict quality gates on the reverse integration process that rolls changes into the mainline, it's possible to successfully herd hundreds of developers in a common direction. Not easy, perhaps, but possible. Tools won't do it all, of course, and you’ll also find a discussion here on topics such as triage and the organizational structure that can make it all happen.

Devil in the Details
This book also digs deep into many of the details of actual build processes at Microsoft that Maraia has been a part of, from Windows NT to Visual Studio. He discusses the way they configure machines, the build tools that have evolved over the years, SNAP builds (Microsoft's own equivalent of continuous integration), and build environments, among other things. It's entirely possible that these details are going to be irrelevant to your own work (while still making for some darned good stories in places -- the physical security of build labs, for example, leads to a very amusing chapter). But overall the details get one point clearly across: The build master is in the business of removing uncertainty from the process.

To a large extent, this reflects the paradoxical nature of the build master's job. If a build master is doing a good job, you should see no evidence at all of their personality: the entire process of producing software should be completely repeatable, working the same way every time. There's no scope for creativity there. Yet to get to that point, you need to do a whole lot of hard work and planning. Even if you don't make exactly the same decisions as Microsoft in setting up the foundations for your build (and you probably won't), these chapters serve as an excellent map of the areas where you'll want to nail things down.

Test, Setup, Ship
In the dim past, you might just have been building the code for your application. Now, of course, things are much more complex than that. The build master has to consider a wide variety of ancillary processes around the core software, including:

• Running build verification tests (BVTs) and smoke tests to make sure that each build passes some rough quality bar.
• Building the setup program that will actually lay down the final bits on the customer’s computer.
• Actually picking a build to release and shipping the darned thing.

Maraia devotes chapters to these activities and others, including internationalization, the quirks of building managed code, and the nuances of picking file version numbers. Each of these chapters can be read as a mini-seminar in its subject. There's not a whole lot of depth to them. The internationalization chapter, for example, isn't going to make you into an expert in internationalizing software but they do serve as quick roadmaps to point you at the major issues.

It’s Only Microsoft
When all's said and done, I think the best way to evaluate this book is to take another look at the subtitle: "Microsoft’s software configuration management best practices." Microsoft has certainly managed to ship a bunch of nifty software, and they've organically grown techniques to do so along the way.

On the other hand, Microsoft isn't the only fish in the software sea, and they've had their share of well-publicized slips along the way (cough Vista cough). If you're applying for a job in a build lab at Microsoft, I'd consider this book required reading. If you're planning to run your own build lab, I'd still say it’s required reading but strictly for input. Consider where your organization resembles Microsoft and where it differs, where your tool stack diverges from Microsoft's, and then develop your own set of best practices. That way you too can ship quality software in a repeatable fashion.