To: Brett Behm who wrote (3644 ) 11/21/1997 11:19:00 PM From: Jojo Mosko Read Replies (1) | Respond to of 14451
Sun goofed on the numbers they published. Enclosed are two messages forwarded from the usenet newsgroup comp.benchmarks. The bottom line is: o Sun is _not_ faster. It is slower o Sun uses deceptive marketing techniques in order to win business from SGI (they are actually _lying_ in that press release) o Sun UE10000 is far more expensive than the Origin for similar CPU and memory configs. o If anything this press release proves that Sun is way behind SGI in performance _and_ price performance. From: mccalpin@frakir.engr.sgi.com (John McCalpin) Newsgroups: comp.benchmarks,comp.sys.super Subject: Sun goofs big on specrate claims Date: 22 Nov 1997 00:26:38 GMT Organization: Silicon Graphics, Inc., Mountain View, CA Lines: 93 Message-ID: <6558ru$noi$1@murrow.corp.sgi.com> Reply-To: mccalpin@engr.sgi.com NNTP-Posting-Host: frakir.engr.sgi.com Xref: fido.asd.sgi.com comp.benchmarks:25422 comp.sys.super:9318 On Yahoo today (11/21/1997), Sun has a press release claiming "Industry-Leading" performance for SPEC95 benchmarks. biz.yahoo.com Too bad it was out of date before it was released.... Here is a table comparing Sun results to the SGI Origin2000 results announced on Monday 11/17/1997: biz.yahoo.com The full SGI press release is at: sgi.com --------------------------------------------------------------- Benchmark System Result No. CPUs --------------------------------------------------------------- SPECint_rate_base95 ******* SGI is 33% FASTER ******** SGI Origin2000 5704 64 Sun HPC 10000 4289 64 SPECint_rate95 ******* SGI is 20% FASTER ******** SGI Origin2000 5922 64 Sun HPC 10000 4945 64 --------------------------------------------------------------- SPECfp_rate_base95 ******* SGI is 68% FASTER ******** SGI Origin2000 9074 64 Sun HPC 10000 5417 64 SPECfp_rate95 ******* SGI is 58% FASTER ******** SGI Origin2000 9478 64 Sun HPC 10000 6013 64 --------------------------------------------------------------- SPECint_rate_base95 Sun HPC 10000 cluster 7339 128 SPECint_rate95 Sun HPC 10000 cluster 8381 128 --------------------------------------------------------------- Note that the Sun Ultra 10000 results were for 4 MB caches, while most (all?) of the Ultra 10000 machines in the field have 1 MB caches. The information available to me says that U.S. list pricing on the Origin2000 is lower for equivalent configurations (equal cpu counts + RAM + I/O). Of course, as the SPECrates show, these would not be configurations of equal *performance*. By clustering together two 64-cpu machines, Sun was able to beat the Origin2000's SPECint_rate by 29% and 42%. The Sun cluster costs about 2.7x as much as the 64-cpu Origin2000, which makes that improvement look a bit less impressive. We have not seen any compelling reasons to do clustered SPECrates so far.... Finally, readers should note that the Sun press release is either the product of mathematical illiteracy or deliberate misrepresentation of the facts: The release claims that the Sun 64-cpu SPECint_rate result is "186% faster" than the SGI result, apparently referring to the SPECint_rate of 2660 on 32-cpus previously published at the SPEC web site. Sorry to rain on the Sun parade, but 4945 for the Sun 64-cpu machine is 1.86 times the (obsolete) result of 2660 on the 32-cpu Origin2000. This is 86% faster, not 186% faster. Using current SGI results of 3173 on 32 cpus, the Sun is only 56% faster, and this is comparing a $2.8M 64-cpu Sun against a $1.0M 32-cpu SGI. Similarly, the clustered SPECint_rate of 8381 is reported to be "315% faster" than the Origin2000, while it is actually 3.15x as fast, or 215% faster. Again, using the current SGI results, the 128-cpu, $5.6M Sun cluster is 2.64 times as fast (164% faster than) the $1.0M 32-cpu SGI Origin2000. So Sun ignores recent results, and materially mis-states performance advantages of systems that are many times larger and more expensive than what they are comparing against. I can see why they have been hiding the Ultra 10000 results for so long, though.... P.S. I am heading out on vacation, and it is likely that I won't bother to respond to flames when I get back.... P.P.S. These are my personal opinions, and do not constitute official statements from my employer. It is likely that I will be reprimanded for my distinct lack of tact here, but the Sun press release was so egregious that I think it is worth the risk. -- -- John D. McCalpin, Ph.D. Supercomputing Performance Analyst Technical Computing Group reality.sgi.com Silicon Graphics, Inc. mccalpin@sgi.com 650-933-7407 From: mccalpin@frakir.engr.sgi.com (John McCalpin) Newsgroups: comp.benchmarks,comp.sys.super Subject: Re: Sun goofs big on specrate claims Date: 22 Nov 1997 01:38:13 GMT Organization: Silicon Graphics, Inc., Mountain View, CA Lines: 26 Message-ID: <655d25$pk3$1@murrow.corp.sgi.com> References: <6558ru$noi$1@murrow.corp.sgi.com> Reply-To: mccalpin@engr.sgi.com NNTP-Posting-Host: frakir.engr.sgi.com Xref: fido.asd.sgi.com comp.benchmarks:25423 comp.sys.super:9319 In article <6558ru$noi$1@murrow.corp.sgi.com>, John McCalpin <mccalpin@engr.sgi.com> wrote: >On Yahoo today (11/21/1997), Sun has a press release claiming >"Industry-Leading" performance for SPEC95 benchmarks. > biz.yahoo.com > >Too bad it was out of date before it was released.... In the spirit of fair play, I should note that the 64-cpu Sun FP_rate results are actually in 4th place: --------------------------------------------------------------- Benchmark System Result No. CPUs --------------------------------------------------------------- SPECfp_rate95 SGI Origin2000 9478 64 IBM SP 120 MHz 8228 64 Hewlett Packard X-Class 6140 64 Sun HPC 10000 6013 64 --------------------------------------------------------------- Hmmm.... An interesting take on "Industry Leading" !!!! -- -- John D. McCalpin, Ph.D. Supercomputing Performance Analyst Technical Computing Group reality.sgi.com Silicon Graphics, Inc. mccalpin@sgi.com 650-933-7407