BSIMM3 Continues To Add Real-World Data to Security Maturity Model

The intrepid trio of app security mavens who decided back in 2009 that it was about time the world had a set of best practices for developing and growing an enterprise-wide software security program based on actual data has unveiled the third version of their innovative Building Security In Maturity Model (BSIMM).

A "maturity model" describes the capability of an organization's processes in a range of areas, from software engineering to personnel management. The Capability Maturity Model (CMM) is a well-known example from software engineering. The BSIMM (pronounced "bee-simm") is the first maturity model for security initiatives created entirely from real-world data.

BSIMM3 which is distributed free under a Creative Commons license, provides insight into 42 of the most successful software security initiatives in the world. The list of companies studied for BSIMM3 includes Adobe, Aon, Bank of America, Capital One, The Depository Trust & Clearing Corporation (DTCC), EMC, Fannie Mae, Google, Intel, Intuit, McKesson, Microsoft, Nokia, QUALCOMM, Sallie Mae, SAP, Scripps Networks Interactive, Sony Ericsson, Standard Life, SWIFT, Symantec, Telecom Italia, Thomson Reuters, Visa, VMware, Wells Fargo and Zynga.

Dr. Gary McGraw, CTO of Cigital; Sammy Migues, director of knowledge management at Cigital; and Dr. Brian Chess, chief scientist at Fortify Software (acquired by HP last year), are the co-authors of this on-going, multi-year study. The purpose of the project, McGraw told me, is to build a "measuring stick," so that companies can compare themselves to companies in their industries who have managed successful software security initiatives. Using the BSIMM measuring stick, McGraw, Migues, and Chess conducted a series of in-person interviews with executives in charge of software security initiatives.

McGraw emphasized that the model is fact-based. "We wanted to turn from the early days of evangelism and advocacy in software security and science," he said. "And this is how to do it."

The project has grown considerably since BSIMM1, which looked at only nine companies. BSIMM3 describes the work of 786 software security professionals working with a satellite of 1,750 affiliated professionals to secure the software developed by 185, 316 developers. The participating organizations represent eight overlapping industry verticals, including: financial services, independent software vendors, technology firms, telecommunications, insurance, energy, media, and healthcare. The current release includes 109 updated activity descriptions and a longitudinal study describing the evolution of eleven of the forty-two firms over time.

BSIMM3 describes 109 activities in 12 practices with 2 or more real examples for each activity. Eleven of the participating firms were measured twice, providing longitudinal study data; those data showed measurable improvement, McGraw said.

The BSIMM3 data set has 81 distinct measurements; some firms were measured twice, while some had multiple divisions measured separately. Among the revelations in this version of the study is the fact that the leading firms on average employ two full-time software security specialists for every 100 developers.

"It's exciting to see something that started out as kind of a backyard science experiment bust out of its test tube and take on a life of its own," McGraw said.

BSIMM3 results conclude that "mature" software security initiatives are "well rounded," with activities in all twelve practices, including: strategy and metrics, compliance and policy, architecture analysis, code review, security testing, penetration testing and configuration management.

"One of the coolest side effects of the project is the community that's growing up around it," McGraw said. "We held a conference last year in Annapolis, and 22 of the 30 firms [attending] sent the executive in charge of software security. We all got together and talked hardcore software security. There's this feeling now of a community of professionals trying to solve the same problems in software security."

For more information and to access the BSIMM3 study, click here.

Posted by John K. Waters on September 30, 2011