Bearded One, many odd things have happened with benchmarks. 20 odd years ago, global optimizing Fortran compilers started getting amazing results on benchmarks, because they could figure out when calculations were being done that were never used for anything, and just didn't do them. Dead code elimination is a legitimate optimization, they were just bad benchmarks.
Just to repeat myself, my reading is that Sun hoisted some class code into the compiler and did some constant folding type calculation there. Constant folding (i.e. doing calculations at compile time if possible, instead of repeating them at program run time) is a standard compiler optimization, and not a particularly sophisticated one. It's dubious to do it as a special case for a particular benchmark, but it's also a sign of a poorly designed benchmark. The technique could even be generalized for arbitrary classes, though I doubt it would be worthwhile. A 50-100 speed improvement in one particular test sticks out like a sore thumb, and if the entire benchmark was scaled so that improvement in one test skewed the results significantly, that's even more indication it's not a very good benchmark.
That's just a guess though. If more news comes in on this, I'll try to track down more info, but if it's a commercial benchmark the source probably isn't publicly available. After the secret formula Java "high end financial math fraud" episode, I'm not going to waste time on things that can't be judged from public information. Sun may well have done something stupid here, for sure. I've never said Sun was without fault, I even credit them as the originators of "embrace and demolish", on the Unix front. But this doesn't look like a federal offense just yet.
Cheers, Dan. |
