To: FJB who wrote (142616 ) 9/2/2001 10:25:59 AM From: pgerassi Read Replies (1) | Respond to of 186894 Dear Bob: Here we go again! SPEC is FLAWED!!!!! Spec contains 25 applications that have not been updated to current algorithms. They do not allow the use of libraries that are used by all other applications and tasks including the ones performed by those 25 applications in the suite. They do not allow the source to change for those applications. Yet how many users alter the code when a new better way is found? One application (program) alone has a 10x speed up from using the ATLAS library for functions contained in the program. Using the geometric average used in the averaging of the output, a 16% increase in the average occurs for that one change. By using these libraries of optimized code, which are available for all processors, a 50% increase in the scores probably would happen from that one change. It also would do to change the relative ranking of the processor types. The second problem with SPEC is that it uses too large a working set for mainstream PCs. It tests the cache and memory subsystem more than raw computing power. This is hurt by the problem above as the constants in those programs seems to optimize for a 4MB direct mapped cache. Use of the libraries dynamically shrinks the block sizes to fit in that processor type's cache size thus, making those huge speed increases and it shows that the difference will grow in the future. The SDRAM, DDRDRAM and dual DRDRAM shows this point as well. The same CPU with different memory bandwidths and latencies, always seems to favor the larger bandwidth over latency. Most tasks, including many of those apps used in the suite, shows that when code is optimized for maximum performance, latency is a far bigger determinant than gross memory bandwidth. Even the benefits of hardware prefetching where bandwidth is traded for reduced latency, shows the overall benefit of reduced latency to task performance. All in all SPEC has become a compiler and main memory bandwidth benchmark. Especially when trying to rate the raw computing power of a processor. It does not reflect real world performance any more. For that, use Tim Wilken's Sciencemark, Anand's RDMS server benchmark, Linux Kernel compile test and IT's Constant Computing tests. Pete