BSIMM-V: Free Software Security Insights from 67 Companies

Here's a provocative statistic: Within a group of leading companies that includes Microsoft, PayPal, Salesforce, Nokia, Sony Mobile, and Visa, the average ratio of full-time software security specialists to developers is 1.4/100. That's one of the findings in the recently published fifth edition of the software-security "measuring stick" known as the BSIMM (Building Security In Maturity Model).

A "maturity model" describes the capability of an organization's processes in a range of areas, from software engineering to personnel management. The Capability Maturity Model (CMM) is a well-known example from software engineering. The BSIMM (pronounced "bee-simm") is the first maturity model for security initiatives created entirely from real-world data.

The BSIMM was developed as a tool to help organizations evaluate their software security programs by comparing them to the programs of other companies. It's based on data collected by its authors through interviews and direct observations of the most successful large-scale software security programs. Although those programs use different methodologies and terminologies, they're described in a uniform way in the BSIMM via a framework, called the Software Security Framework, which provides a common vocabulary and allows for apples-to-apples comparisons.

So, is that ratio of software security pros to developers the right one? That's not a question the BSIMM was designed to answer, says one of its authors.

"The BSIMM is based on the study of real practices as they exist," explained Gary McGraw, CTO of security consulting firm Cigital and author of eight books on software security. "It describes those practices; it's not a prescriptive model. But it's real data, not hunches and guesses, so I can go to the board and say, here's you, and here are the other 26 firms that look like you that we've measured before. And I can say, it looks like you're the slowest zebra. And then we have a conversation about that."

BSIMM-V includes data from 67 participating companies, up from the 51 included in the fourth edition. The number of companies has grown every year since the first edition was published in 2008; that one was based on studies of nine software security initiatives. BSIMM-V describes the work of about 3,000 people, collectively, McGraw said.

As a measuring stick, the BSIMM allows an organization to compare and contrast its own software security efforts with those of its peers. As the report puts it, "You can then identify goals and objectives of your own and look to the BSIMM to determine which additional activities make sense for you."

BSIMM's authors argue that highly mature initiatives are well rounded, carrying out all of the 12 core practices described by the model, including: strategy and metrics, compliance and policy, architecture analysis, code review, security testing, penetration testing, and configuration management. The model also describes how mature software security initiatives evolve, change, and improve over time.

During the course of their investigation, the researchers have observed a total of 112 activities related to software security. These are actions carried out or facilitated by the software security group within an organization as part of a practice, and each activity is directly associated with an objective. The researchers added two new activities in the last edition of the BSIMM based on their observations in the field: simulate software crisis and automate malicious code detection. BSIMM-V adds another new activity: operate a bug bounty program.

Keep in mind that what the BSIMM is describing is security activities around software development, specifically. The computer security industry as a whole is growing fast, McGraw noted, at a rate of about 8.9% per year, generating between $20 and $40 billion in revenue annually. And while software security accounts for only 10% of that growth, he said, that segment is growing more than twice as fast: 20% per year, by some estimates. "I like to think of us finally as the pinky fingers on the two hands of computer security," McGraw said.

The BSIMM was originally developed by Cigital and Fortify Software (since acquired by HP). The two most recent editions of the study were authored by McGraw; Sammy Migues, Director of Knowledge Management and Training at Cigital, and Jacob West, CTO of Fortify Products in HP's  Enterprise Security group. The first three BSIMMs were authored by McGraw, Migues, and Brian Chess, distinguished technologist at HP (and co-founder and former chief scientist at Fortify).

BSIMM-V is available for download. It is distributed free under the Creative Commons license.

Posted on December 12, 2013