Subject: Low Level Benchmarks

From: Adolfy Hoisie <[email-masked]>
Date: Fri, 16 Jan 1998 11:14:30 -0700 (MST)

Just to amplify some of the numerous excellent points made by Pat and
Charles and Rolf, the emphasis of the Parkbench group, as I see it, should
be on defining the methodology for benchmarking at this level. A string of
numbers says very little about machine performance in absence of a solid,
scientifcally defined underlying base for the programs utilized for
benchmarking.
COMMS is obsolete in methodology, coding and generation and analysis of
results. As such, I have used it quite some time ago only to reach the
conclusions above. Instead, I always chose to write my own benchmarking
programs in order to extract meaningful data for the applications I was
working on.
I would like to see the debate heading towards what is it that we need to
measure in a suite of general use that is applicable to machines of
interest. For example, very little or no attention is being paid to
benchmarking DSM architectures, where quite a few architectural parameters
become harder to define and subtler to interpret. Including, but not
limited to, message passing characterization on these architectures.

Adolfy    

======================================================================
Adolfy Hoisie                  \        Los Alamos National Laboratory
                                \Scientific Computing, CIC-19, MS B256
[email-masked]                  \           Los Alamos, NM  87545 USA
                                  \                Phone: 505-667-5216
http://www.c3.lanl.gov/~hoisie/hoisie.html           FAX: 505-667-1126