It is well-known that I am a long-time user of Apple Macintosh computers, and I am very leery of Microsoft Windows and Linux because of the many security problems that continue to plague them.  (However, I use Windows, and Linux, and Solaris, and a number of other systems for some things—I believe in using the right tool for each task.)  Thus, it is perhaps no surprise that a few people have written to me with a “Nyah, nyah” message after reading a recent article claiming that Windows is the most secure OS over the last six months. However, any such attitude evidences a certain lack of knowledge of statistics, history, and the underlying Symantec report itself.  It is possible to lie with statistics—or, at the least, be significantly misled, if one is not careful.

First of all, the news article reported that —in the reporting period—Microsoft software had 12 serious vulnerabilities plus another 27 less serious vulnerabilities.  This was compared with 1 serious vulnerability in Apple software out of a total of 43 vulnerabilities.  To say that this confirms the point because there were fewer vulnerabilities reported in MS software (39 vs. 43) without noting the difference in severity is clearly misleading.  After all, there were 12 times as many severe vulnerabilities in MS software as in Apple software (and more than in some or all of the others systems, too—see the full report).

Imagine reading a report in the newspaper on crime statistics.  The report says that Portland saw one killing and 42 instances of littering, while Detroit had 27 instances of jaywalking and 12 instances of rape and homicide.  If the reporter concluded that Detroit was the safer place to live and work, would you agree?  Where do you think you would feel safer?  Where would you be safer (assuming the population sizes were similar; in reality, Portland is about 2/3 the population of Detroit)?

More from a stochastic point of view, if we assume that the identification of flaws is more or less a random process with some independence, then it is not surprising if there are intervals where the relative performance in that period does not match the overall behavior.  So, we should not jump to overall conclusions when there are one or two observational periods where one system dominates another in contrast to previous behavior.  Any critical decisions we might wish to make about quality and safety should be based on a longer baseline; in this case, the Microsoft products continue to be poor compared to some other systems, including Apple.  We might also want to factor in the size of the exposed population, the actual amount of damages and other such issues.

By analogy, imagine you are betting on horses.  One horse you have been tracking, named Redmond, has not been performing well.  In nearly every race that horse has come in at or below the middle of the pack, and often comes in last, despite being a crowd favorite.  The horse looks good, and lots of people bet on it, but it never wins.  Then, one day, in a close heat, Redmond wins!  In a solid but unexciting race, Redmond comes in ahead of multiple-race winner #2 (Cupertino) by a stride.  Some long-time bettors crow about the victory, and say they knew that Remond was the champ.  So, you have money to gamble with.  Are you going to bet on Redmond to win or place in each of the next 5 races?

Last of all, I could not find a spot in the actual Symantec report where it was stated that any one system is more secure than another—that is something stated by the reporter (Andy Patrizio) who wrote the article.  Any claim that ANY system with critical flaws is “secure” or “more secure” is an abuse of the term.  That is akin to saying that a cocktail with only one poison is more “healthful” than a cocktail with six poisons.  Both are lethal, and neither is healthful under any sane interpretation of the words.

So, in conclusion, let me note that any serious flaws reported are not a good thing, and none of the vendors listed (and there are more than simply Microsoft and Apple) should take pride in the stated results.  I also want to note that although I would not necessarily pick a MS platform for an application environment where I have a strong need for security, neither would I automatically veto it.  Properly configure and protect any system and it may be a good candidate in a medium or low threat environment. As well, the people at Microsoft are certainly devoting lots of resources to try to make their products better (although I think they are trapped by some very poor choices made in the past).

Dr. Dan Geer made a riveting and thought-provoking presentation on cyber security trends and statistics as the closing keynote address of this year’s annual CERIAS Security Symposium.  His presentation materials will shortly be linked into the symposium WWW site, and a video of his talk is here.  I recommend that you check that out as additional material, if you are interested in the topic.