“PayPal transacts more than US$1,500 every second of every day, with millions of people around the world relying on the robustness of its system.

It comes as a surprise to many people that PayPal runs such a large financial services company on an open source platform, but that’s precisely how we’re able to deal with the two competing demands our business model places on us: security and innovation.

Thanks for taking the time to respond. Unfortunately, I don’t feel that your response provides the balance I’m looking for. I certainly agree that MS have significantly improved the level of security in their products in recent years. I commend them for doing so. However, I’m more than a little concerned that your unsupported claims of superiority are not helpful.

Please understand, I am not saying your claims are wrong, only that they are unsupported.

Specifically, you say:

“I have found no evidence that IBM, Intel, and RedHat train their people in threat modeling, for example.”

Have you actually looked for such evidence beyond the reviews on the howsoftwareisbuilt.com website? If so what depth of analysis have you made?

I decided to take your challenge and look at the interviews that mention security on the howsoftwareisbuilt site. My findings are summarised below. Note, I’m ignoring interviews with MS folk as I think we can all agree that MS have significantly improved their security QA processes in recent years. The question here is whether we can justify a claim that other companies do not have similarly robust security QA processes.

In the MySQL interview it is clearly stated that the many eyes principle is what makes things secure.

Stormy Peters is asked about the MySQL comment specifically, but does not directly address it in her answer.

Mark Gross (Intel) states that many eyes is important, but clarifies by saying that this means “You get people testing it.” Unfortunately, he does not expand on what he means by “testing it”, he could be referring to an ad-hoc process, he could be referring to a very dilligent internal process.

In the PostgresSQL interview Josh Berkus does not refer to the many eyes idea directly, but he observes that visible code is clean of “mystery functions that nobody understands”, thus fixes are easier to implement if a vulnability is found.

That’s it. The total number of projects specifically interviewed on this topic is one (MySQL), whilst three other people talk about it in some way. None of these companies are the ones Scott refers to in the previous comment or his original post.

Perhaps howsoftwareisbuilt.com would like to interview the companies and/or projects named. I’d be really interested to hear responses to the original blog post or to the following questions:

What do Red Hat do that enables them to claim that:

“Red Hat continually looks for potential security exposures and delivers tested security updates through Red Hat Network”

What do Oracle do that enables them to claim that:

“We have a complete Oracle and Linux security support, and offer support for Oracle Linux security tools”

What do IBM do that enables them to claim that:

“IBM will help you assess, detect, protect, correct and recover from security exposures in your IT and physical security environments in today’s on demand world.”

My point remains the same as in the original post. All three companies (and many others) bet their business, at least in part, on security. I cannot accept that they do so without having internal processes to manage the assocaited risks and I cannot accept that those processes do not feed back into the open source projects they use in their businesses.

Finally, Scott, you say “the data on known vulnerabilities seems to indicate that Microsoft products developed after the inception of their “Security Development Lifecycle” have fewer known vulnerabilities than the open-source counterparts.”. Can you please point us at this data. I am currently unaware of any reliable, unbiased and extensive data covering this topic area. As an unbiased, non-advocacy advisory service we would find such data invaluable.

I agree, it’s hard to fathom that companies like IBM, Sun, Oracle, RedHat, etc, wouldn’t really focus on secure coding. It’s hard to believe that fortune 500 companies would be running an operating system that hasn’t gone through extensive security testing, but so far, I haven’t uncovered significant evidence to the contrary.

I have found no evidence that IBM, Intel, and RedHat train their people in threat modeling, for example. APIs like strcpy haven’t been banned from the kernel or apache development. If someone like IBM does write test suites to look at the security of sub-systems, I don’t see that those security specific tests have been checked in for others to run and improve on. There are tests for specific security features, but those are not the same as security focused tests across all the subsystems.

Again, maybe I just haven’t talked to the right people, and if so, I’d love to correct my stance, but based off a substantial number of interviews (you can look at the list of people interviewed so far at http://www.howsoftwareisbuilt.com), I’m left with the conclusion that Microsoft is leading the industry when it comes to building secure products. While I can’t comment on the number of unknown vulnerabilities, the data on known vulnerabilities seems to indicate that Microsoft products developed after the inception of their “Security Development Lifecycle” have fewer known vulnerabilities than the open-source counterparts.