Re: I agree that it would be a great idea to establish a 3rd party standard for labeling performance, but I also think that there are many difficulties involved. Bapco was supposed to be such a 3rd party, but would you agree that they can fairly and accurately label performance?
Well, let's see. Bapco is funded by Intel, did that affect their benchmarks? For sysmark 2001, they found a program with a bug in it that gave P4 huge advantage. So they ran that program in the background, for all the other benchmarks, making P4 look better in all the benchmarks, not just the one where P4 had an advantage due to a known bug.
For Sysmark 2002, Bapco stopped letting anyone know what was actually being run, so that they couldn't be caught forging false results as easily as they were caught with sysmark 2000.
So my answer is that no, I wouldn't agree that Bapco fairly and accurately labels performance.
I like the Tim Wilkens / U of I Supercomputer Center Sciencemark because it's open source which stops swindlers like Bapco from faking results with it.
w3.physics.uiuc.edu
It has been proven that nothing but open source benchmarks can be relied upon - other benchmarks, like Sysmark, are primarily used by cheaters and frauds like Intel. |
