Microsoft's mythical man-years
The company boasts that it's making Herculean security efforts -- but throwing more people at software problems rarely solves them.
salon.com
"I'd be astonished," said Steven B. Lipner, Microsoft's director of security assurance, "if the open-source community has in total done as many man-years of computer security code reviews as we have done in the last two months."
...
Technically, Lipner is saying the following: Let X equal the number of individual Microsoft programmers reviewing its products' security, multiplied by the amount of time each has spent on the task. Let Y equal the number of open-source programmers reviewing their software's security, multiplied by the amount of time they have spent on the task. X is way greater than Y. All this rings with the kind of scary precision that cows nontechnical people when they hear it in engineers' voices.
The trouble is, the whole concept of measuring software productivity in "man-years" or "man-months" is profoundly discredited -- and not by some radical new theory of software development, but in what is probably the single most seminal work on software management: Frederick P. Brooks' "The Mythical Man-Month," first published in 1975, when Bill Gates was a stripling and personal computing a dream.
Brooks was an IBM veteran who'd watched Big Blue's mainframe software projects spiral out of control in the 1960s. As he analyzed the company's epic failures -- which earned the label of "software crisis" in their day -- he discovered a counterintuitive principle: "Adding manpower to a late software project makes it later."
How can that be? Brooks argued that, with most common large-scale software projects, adding manpower to a team results in further delays, as veterans stop to introduce newcomers to the complexities and challenges of the project, and as managers step back to divide up the work afresh. When a team is behind schedule, throwing new people at the problem actually makes the problem worse. Brooks concluded, "The man-month as a unit for measuring the size of a job is a dangerous and deceptive myth."
While most aspects of the software business have changed since 1975, and good practices have helped many development efforts skirt the kinds of disasters Brooks observed at IBM, the general validity of his observation remains unchallenged. Which might leave you wondering what a Microsoft manager is doing, in 2002, boasting about how many man-years his company is throwing at its current top-priority project.
...
Roy Fielding, a Web pioneer who helped create the Apache Web server used by the majority of publicly accessible Web sites (it's serving the page you're reading) and is now chairman of the Apache Software Foundation, points out that one root of Microsoft's security woes lies in its development process itself -- which "encourages individuals to make large changes to the products under deadline pressure, without adequate peer review of every single change at the time it is made." Open source works differently: "Every change that is made to the Apache code bases is ... posted to a mailing list where any person who wants to review changes can do so, in public, and the first person who identifies a potential security problem in a change is given instant credibility within the community."
In this view, the total number of "man-years" of security code review is largely irrelevant. No matter how smart Microsoft's developers may be, they are all part of one company's culture, and the odds are good that no matter how many hours they spend improving their code, they will not collectively be able to imagine all the myriad ways the entire universe of computer users -- and mischief-makers -- will attempt to break it.
Fielding says that Lipner's "more man-years" claim is "absolute crap. They probably spent more money on it, but he is misdirecting the public based on the theory that there are fewer open source developers per project than there are people per project within Microsoft. Open source developers are only a small subset of the people who do security reviews of open source code. Most open source security reviews are done by the hackers and security consultants that make a living from finding (and sometimes exploiting) security holes. They have a very strong incentive for publishing their findings."
The open-source model, in other words, allows for a kind of global stress-testing, peer review and transparent repair that Microsoft can never guarantee. Since its code is proprietary, you can only take Microsoft's word that it has fixed bugs and plugged security holes. And the next time a rogue virus takes down your company's e-mail server, all you can do is curse -- and wait for the company to issue a fix.
Today, with the software industry a linchpin of the global economy, we tend to think of open source as a radical new challenge to the Microsoft-style norm. So it's useful, in looking back at a classic like Brooks' "Mythical Man-Month," to be reminded that -- in the days before Gates and company built their empire on operating-system software -- open source was once considered simple common sense.
In Brooks' day, a program had no general value -- was not considered a true "programming product," in Brooks' words -- unless it could be "run, tested, repaired, and extended by anybody." (The italics are mine.) Such programming "products" require "thorough documentation, so that anyone may use it, fix it, and extend it." To Brooks, and many other software experts of his era, if the programmer hadn't enabled anyone to fix or extend his work, he hadn't finished his job.
Microsoft takes a different view -- always has. With its vast resources, Microsoft can afford more "man-years" than anyone else on earth. But can it rewrite principles of the software business first identified nearly 30 years ago?
The answer will become plain as the results of the "trustworthy computing" project emerge. If the torrent of security gaffes in Microsoft products vanishes, we can applaud Redmond's intrepid troops. But if we're still battling the spawn of the NIMDA and Code Red worms in a year or two, it's time to stop trusting Bill Gates for good.
I've ordered CodeWeavers CrossOver Office. It sounds like business can save MILLIONS and I can keep my beloved "Excel".
Best of luck.
Microsoft ... got TOAST? |
