(LONG) Remember that any system-based performance metric will be open to rampant cheating, or the accusations of such.
After all, look at how the "sour grapes" AMDroids continually rant and rant over Intel's alleged cheating on SPEC. The average consumer has absolutely no clue what SPEC is, so the motivations for the alleged cheating is limited. But imagine if you have a performance rating that is high-profile and is able to influence the average consumer on buying decisions. You'll have every boxmaker, from the big Tier 1 guy to the mom-n-pop shop, do whatever it takes to fudge the numbers. Oh, and don't forget about the component manufacturers, from graphics cards to hard drives to even memory, who also stand to benefit from fudged numbers. The performance rat race will have extended well beyond Intel and AMD.
One way to prevent such cheating is to make sure the suite of benchmarks is easily runnable by the average consumer. That will keep the system builder honest, for if the customer can't reproduce the performance numbers, the company should have a lot of explaining to do. But how's the consumer supposed to respond if the company says, "Oh, we disable just about every background task imaginable, including virus checkers and Instant Messaging. Just try that and you'll get the performance we expect"? Or how are the companies supposed to respond to consumers who complain about a steady degradation of performance over time? Geeks like us know how to do a clean install of the software, but do the consumers?
One thing I like about MHz is the fact that you cannot easily fake it. Even a processor designed with high clock speed in mind (like the Pentium 4) has many challenges to meet, including process limitations and power dissipation. But when you buy a processor that runs at a given clock speed, you KNOW how fast that processor is running. Whether that actually translates to real performance is another question. But at least MHz is not as easily fudged as, say, QuantiSpeed (whose scale is only determined by the whims of marketeers).
This probably sounded like a long-winded defense of MHz as a (if not THE) metric of performance, and that wasn't my intention. My point is that any high-profile metric of performance will be subject to distortion and politics, no matter how objective you make it or how trustworthy the players are. And we'll only be trading one debate for another.
Tenchusatsu |
