To: Joe NYC who wrote (104441 ) 6/14/2000 11:58:00 AM From: pgerassi Read Replies (2) | Respond to of 186894
Dear Joe: I think that a majority of performance users use C or C++ as the front end to a process that includes profiling. The output of the profiler is to determine which parts of the code are the most used and therefore optimizing them will have the highest impact in the performance of the application. In these areas, assembly work is done to tighten up the code to get the highest performance. Here is where there is no substitute for assembly programming. But most commercial applications can not work on just one processor type. The application must run on Katmai, Celeron, Coppermine, and now Athlon. They usually program for the highest common denominators. Processor specific optimizations would shrink the market. For most high end workstation uses, multiple processor families are also included, SPARC, RS/6000, Alpha, PA-RISC, as well as x86. Also most developers want three additional things, stability, repeatability, and accuracy. Games allow some problems (even want) in the last two but, simulations and real work requires all three. From the reports I have seen, Intel's VFC 4.5 compiler seems to lack all three in parts. I can not cite news source links because I am not on my home machine and even there, I clean out space on a regular basis. Besides, most such links get purged from the sites for the same reasons. The link I know still having commentary on the use of VFC is on Tim Wilkin's Scientific Benchmark page found in the news item of 6/12/2000. Even this will soon be placed in an archive and probably purged at some data. (I find it ethical to say how a page was found so that those that gather the information can be cited) The need for accuracy is why most users do not try to eke out the highest performance and use statistical sampling to insure that the output is correct and not the result of some particular artifact or bug. That is why some routines are run using a different algorithm, sometimes on different hardware. This, I learned early in Numerical Methods class I took as a college freshman. It has served me well. Loss of significance is one of the most insidious problems with these type of applications. You can lose it without even realizing it. And any output you get is totally useless. How do you think the FDIV bug was found? The other problem with typical benchmarks is the very thing that makes them easy to run and use, they are fixed. Here is where cheating occurs. The original intent of SPEC was to mimick the environment of typical workstation uses. The rules were put in place to restrict cheating. They have thus failed. All the major companies except for AMD use speciallized compilers just for this benchmark. The original intent was for it to be used as AMD first used it. Get your platform, get the available commercial compilers for that machine, find one that generates the quickest accurate code by testing and "playing around", and submit the best scores obtained overall with the same set of compiler settings. All of the others cheat to some extent over the spirit of the benchmark. This may be due to resources, etc. But at least in these cases, AMD followed the spirit of the suite. Now given how this is really used, I think that SPEC, the organization, should generate a new benchmark, tighten the rules, and require that a public domain compiler be used. Any company can rework the compiler but, all the changes must be in the public domain to allow anyone to verify that no shenanigans are used. Now the results would be in the spirit of the original benchmark. This is no longer unworkable. (The benefits of public domain software) Pete