To: Joe NYC who wrote (87504 ) 8/23/2002 2:24:57 PM From: Petz Read Replies (1) | Respond to I disagree Van on the idea that there is something wrong with weighting by runtime, if it is done right, although there is no question that Sysmark 2002 vs. 2001 proves bias by their choice and weighting of benchmarks.square of runtime. If true, that is absolutely ludicrous. Runtime-weighting is fine as long as the various repetition counts are chosen so that the runtimes have the desired weight. For example, the desired weight for Office aps might be 25% word processor, 15% database, 20% spreadsheet, 20% presentation and 20% internet browsing. The various tasks within these applications should also be reasonably divided according to usage patterns, and if tasks are too short for this to be possible, the should be repeated enough to get their proper time-weighting. There is nothing wrong with time-weighting if the weights reflect real-world usage statistics. To do it right, Bapco also needs to use both AMD and Intel processors. Perhaps a good set of 5 CPUs to use would be 3 P4's with SDR, DDR or RDRAM, one Celeron SDR machine and one Athlon XP with DDR. The setting of overall weightings should be based on usage questionnares. The result of this should be a list of task selections and weigtings. Repetitions should be used to make the actual benchmark times give runtimes proportionate to the predetermined weightings, when run over a variety of machines using both Intel and AMD processors in blind tests. In other words, the "rep count selectors" only find out the total % contribution of each task and application averaged over all the machines (Intel and AMD), which they can then use to adjust their rep counts.
