News

Report: Vista Has Fewest Vulnerabilities at Six-Month Mark

• By Keith Ward
• June 21, 2007

A "6 month vulnerability report" released Thursday by Microsoft shows that, compared with the first six months following the release of Windows XP, OSes from various Linux distributions, and even Mac OS X 10.4, that Vista is the hands-down winner for fewest security holes.

The report was written by Jeff Jones, a Security Strategy Director in Microsoft's Trustworthy Computing group. He noted that for Vista's first six months (it was released to business on Nov. 30, 2006), a total of 12 vulnerabilities affected Vista. Microsoft rated five of those vulnerabilities as "critical," six as "important" and one did not have a severity rating.

Another organization, The National Institute of Standards (NIST) in the National Vulnerability Database (NVD), had a slightly different rating of the flaws. It rated 10 of the issues as "high" severity, one as "medium" and one as "low."

Jones compared those figures with the vulnerabilities in the first six months for the following OSes: Windows XP, Red Hat Enterprise Linux 4 WS, Ubuntu 6.06 LTS Desktop, Novell SuSE Linux Enterprise Desktop 10 and Mac OS X 10.4 (Tiger). Jones said he picked those particular Linux distributions because they were either very popular (Red Hat and Novell), or an up-and-comer (Ubuntu, which Dell ships as the default distro on its Linux-based computers).

Windows XP, which shipped on Oct. 25, 2001, had 36 vulnerabilities fixed in the first six months, including 23 that the NIST rated as "high" severity. Thus, XP had three times the number of security holes as Vista.

But XP still fared much better than the Linux OSes. Jones compared Vista with two different types of each Linux distribution: a full install with all of the components, and a "stripped-down" version with only those components that make it comparable with Vista functionality. Linux, unlike Microsoft desktop OSes, allows for piecemeal installation of components.

During the first six months following the release of Red Hat Enterprise Linux 4 WS, Red Hat fixed 214 vulnerabilities in the "reduced" version Jones used for comparison, including 62 that the NIST rated as "high" severity.

Novell's SuSE Linux Enterprise Desktop 10 fared better, with 123 flaws in the reduced functionality version fixed by Novell, including 44 rated as "high" severity by the NIST.

Ubuntu Linux came in squarely in the middle of the Linux group.

"During the first six months, Ubuntu fixed 145 vulnerabilities affecting Ubuntu 6.06 LTS. Forty-seven of those fixed were rated 'high' severity in the NVD," Jones noted.

Even Apple, which makes a big show of its security superiority over Windows, fared worse, according to Jones' statistics. He reported that with the first six months of its release, Mac OS X had 60 holes fixed, 18 of which the NIST rated as "high" severity.

Jones' conclusion after looking at the data?

"In all four cases studied for the six-month period after ship, Windows Vista appears to have a lower vulnerability fix and disclosure rate than the other products analyzed, including the reduced Linux installations. This affirms the early results that we found after 90 days and provides a supporting indicator that the Microsoft Security Development Lifecycle process and heightened focus on security is having a positive impact on Microsoft Windows in terms of fewer vulnerabilities."

Not everyone is as convinced, however. Michael Cherry, of independent analyst company Directions on Microsoft, cautioned not to read too much into the figures.

"It's meaningless," he said. "I don't understand this obsession with the number, as if that's a meaningful metric."

Cherry said that the past doesn't necessarily correlate with the future. "As of today, they've looked at six months of Vista, but tomorrow they could be hit by a massive vulnerability, so does this have any predictive value going forward?"

Russ Cooper, a senior analyst with security vendor Cybertrust who also writes for 1105 Media, agreed.

"Looking at desktop security from this perspective is useless. The question is whether I'm going to have compromised malware on my system or not. It's very, very clear that threats exist almost exclusively in the Windows world, that attacks happen almost exclusively in Windows."

Cherry is also suspicious of the less-than-scientific method of determining vulnerabilities.

"Many problems in operating systems are reported by users over time. I'm not convinced there's enough eyes looking at Vista yet."

That doesn't mean that Cherry thinks Vista is insecure, or that Microsoft doesn't take security seriously.

"Do I think Microsoft is doing a better job with security? Absolutely. Are they getting better all the time? Absolutely ... But in this business you don't live and die by how good you're doing, but the last time you messed up. This just seems to be an attempt to build Vista momentum."

Cooper said that six months isn't enough time to determine how secure Vista is.

"We still have very few deployments and Vista-specific applications, compared to those [apps] that are Vista-compatible. We don't have software that uses the new programming model and leverages all these features that are new. For all we know, there's a fundamental flaw in there" that hasn't been discovered yet, and won't until more users are working with more programs, Cooper commented.

Cherry shared the assessment that more time is needed.

"We're talking about an OS that, in essence, has a 10-year life (five years of mainstream support, and five years of extended support). After six months, you're trying to draw a trend line. In a year, you might have enough data to start to think about how it's doing."