Did anyone catch this SNAFU in Anand's 2GHz P4 review?
[Graph showing 1.4GHz Athlon beating 2GHz P4 in Flask MPEG encoding]
Another newcomer to our test suite is MPEG-4 encoding using Flask. We used the MMX iDCT
algorithm for all of the platforms (on the Pentium 4 the SSE2 iDCT algorithm did not perform any
better). Using the 1.1 patched beta of v0.060 and the DivX 4.0 codec we ran an encode at 352
x 288 of a MPEG-1 source with de-interlacing enabled.
(source: anandtech.com )
This is completely unfair to the P4. Anyone who has studied the subject, even casually, should know that the MMX iDCT doesn't give as accurate results as the SSE2 (or any of the other) iDCT. This lack of accuracy translates directly to a loss in image quality.
The thing is, there is some leeway in the standards, allowing the iDCTs to be rather inaccurate - to a certain limit, of course. This allows for greater flexibility in terms of platforms and performance - certainly a Good Thing(tm) - but does make it somewhat harder to use for benchmarking (the risk of comparing apples to oranges).
Flask MPEG actually includes a small feature that allows you to test the accuracy of your selected iDCT. It would be quite easy to see if SSE2 provides the same accuracy as x87, MMX, 3DNow! etc. With this information included, the reviewer could then present benchmarks with platform-dependent iDCTs (making sure the accuracies are very similar in each comparison).
(It should be noted that Anand isn't the only one making this mistake. Tom has some rather dubious benchmarks of this type as well, where he does not even specify which iDCT he uses, making his results rather useless).
One really interesting thing in Anand's test, however, is that the 1.2GHz AthlonMP trumps the 1.4GHz Tbird by 10%.
Flask encoding scales very well with processor frequency (roughly 95% of ideal scaling). Using this, a 1.4GHz AthlonMP would be a massive 20% faster than a identically clocked Tbird. This would do quite a bit to improve AMD's performance in the P4's best area. Of course, AMD needs to actually release a faster Palomino first…
-fyo |
