Subject: Draft Report Chapter-2

From: [email-masked]
Date: Fri, 30 Apr 93 15:09:09 GMT

FIRST DRAFT LOW-LEVEL CHAPTER
                   -----------------------------

I have put together below my thoughts on the contents of the low-level
benchmarks, which are mostly what we have at Southampton and in my
own work. This is for comment and to merge with contributions from
other subcommittee members (none in yet), and finally for presentation to 
24 May 1993 meeting of PBWG.
     I believe we need basic multi-processor benchmarks to measure
various global operations in addition to SYNCH1 (e.g. global sum and 
multiply reduction, global 'or' and 'and', etc). Perhaps other members have 
these or would like to write benchmarks and suggest appropriate metrics 
to use.
     I heard some talk about the alpha and beta parameters at the last
meeting. Can someone tell me what they measure and send me the appropriate
reference. Thank you,
                   Yours sincerely
                         Roger Hockney 
            (leader low-level benchmark subcommittee)
There follows latex version of draft of low-level chapter
------------------------  cut here  -----------------------------------------
%file: lowlev1.tex
\chapter{Low-Level Benchmarks}
\footnote{assembled by Roger Hockney for low-level subcommittee}

\section{Single-Processor Benchmarks}

\subsection{Timer resolution: TICK1}

TICK1 measures the interval between ticks of the clock being used
in the benchmark measurements. That is to say the resolution of
the clock. A succession of calls to the timer routine are inserted
in a loop and executed many times. The differences between successive
values given by the timer are then examined.  If the changes in the
clock value (or ticks) occur less frequently than the time taken to 
enter and leave the timer routine, then most of these differences
will be zero. When a tick takes place, however, a difference equal 
to the tick value will be recorded, surrounded by many zero differences. 
This is the case with clocks of poor resolution, for example most UNIX 
clocks that tick typically every 10 ms. Such poor UNIX clocks can still
be used for low-level benchmark measurements if the benchmark is
repeated, say, 10,000 times, and the timer calls are made outside this 
repeat loop.  

With some computers, such as the
CRAY series, the clock ticks every cycle of the computer, that is to say
every 6ns on the Y-MP. The resolution of the CRAY clock is therefore
approximately one million times better than a UNIX clock, and that is
quite a difference! If TICK1 is used on such a computer the difference
between successive values of the timer is a very accurate measure of
how long it takes to execute the instructions of the timer routine, and
therefore is never zero. TICK1 takes the minimum of all such differences,
and all it is possible to say is that the clock tick is less than or
equal to this value. Typically this minimum will be several hundreds
of clock ticks. With a clock ticking every computer cycle, we can make
low-level benchmark measurements without a repeat loop. Such measurements
can even be made on a busy timeshared system (where many users are
contending for memory access) by taking the minumum time recorded from
a sample of, say, 10,000 single execution measurements. In this case,
the minimum can usually be said to apply to a case when there was no
memory access delay caused by other users.

TICK1 exists and forms part of the Genesis benchmarks v2.0 and v2.1.1
 ~\cite{Hey91}.

\subsection{Timer value: TICK2}

TICK2 confirms that correctness of the time values returned by the
computer clock, by comparing its measurement of a given time interval
with that of an external wall-clock (actually the benchmarker's 
wristwatch). Parallel benchmark performance can only be measured
using the elapsed wall-clock time, as the objective of
parallel execution is to reduce this time. Measurements made with a
CPU-timer (which only records time when its job is executing in the CPU) 
are clearly incorrect, because the clock does not record waiting time 
when the job is out of the CPU. TICK2 will immediately detect the 
incorrect use of a CPU-time-for-this-job-only clock. An example
of a timer that claims to measure elapsed time but is actually a
CPU-timer, is the returned value of the popular Sun UNIX timer ETIME. 
TICK2 also checks that the correct multiplier is being used in the 
computer system software to convert clock ticks to true seconds.  

TICK2 exists and will form part of the next release of the Genesis 
benchmarks v2.2 ~\cite{Hey91}.

\subsection{Most Reported Benchmark: LINPACKD (n=100)}

This well-known standard benchmark is a Fortran program
for the solution of (100x100) dense set of linear equations by
Gaussian elimination. It is distributed by Dr J. J. Dongarra of
the University of Tennessee. The results are quoted in Mflop/s
and are regularly published and available by electronic mail.
The main value of this benchmark is that results are known for 
more computers than any other benchmark. Most of the compute time 
is contained in vectorisable DO-loops such as the DAXPY (scalar 
times vector plus vector) and inner product. Therefore one expects 
vector computers to perform well on this benchmark. The weakness 
of the benchmark is that it tests only a small number of vector 
operations, but it does include the effect of memory access and
it is solving a complete (although small) real problem. 

\subsection{Performance Range: The Livermore Loops}

These are a set of 24 Fortran DO-loops (or kernels) 
extracted from operational codes used at the Lawrence Livermore 
National Laboratory \cite{Ma88}. They have been used since the early 
seventies to assess the arithmetic performance of computers and their 
compilers. They are a mixture of vectorisable and non-vectorisable
loops and test rather fully the computational capabilities of the
hardware, and the skill of the software in compiling efficient code,
and in vectorisation. The main value of the benchmark is the range
of performance that it demonstrates, and in this respect it 
complements the limited range of loops tested in the LINPACK benchmark. 
The benchmark provides the individual performance of each loop,
together with various averages (arithmetic, geometric, harmonic)
and the quartiles of the distribution. However, it is difficult to
give a clear meaning to these averages, and the value of the benchmark
is more in the distribution itself.  In particular, the maximum and
minimum give the range of likely performance in full applications.
The ratio of maximum to minimum performance has been called the
{\em instability} or the {\em speciality} ~\cite{Hoc91}, and is a measure 
of how difficult it is to obtain good performance from the computer,
and therefore how specialised it is. The minimum or worst performance
obtained on these loops is of special value, because there is much
truth in the saying that "the best computer to choose is that with 
the best worst-performance".

\subsection{Basic Arithmetic Operations: RINF1}

This benchmark takes a set of common Fortran DO-loops and analyses
their time of execution in terms of the two parameters \rnhalf
~\cite{Hoc77,HoJe81,Hoc82,Hoc83,Hoc87,HoJe88}. \rinf is the 
asymptotic performance rate in Mflop/s which 
is approached as the loop (or vector) length ,$n$, becomes longer. \nhalf 
(the half-performance length) expresses how rapidly, in terms
increasing vector length, the actual performance, $r$, approaches \rinf. 
It is defined as the vector length required to achieve a performance 
of one half of \rinf. This means that the time, $t$, for a DO-loop
corresponding to $v$ vector operations (i.e. with $v$ floating-point 
operations per element per iteration) is approximated by:
\begin{equation}
                  t = v * ( n + \nhalf ) / \rinf          \label{Eqn1}
\end{equation}      
         Then the performance rate is given by
\begin{equation}
        r =  \frac{v*n}{t} = \frac{\rinf}{(1+\nhalf /n)}   \label{Eqn2}
\end{equation}                                        
We can see from Eqn.(\ref{Eqn1}) that \nhalf is a way of measuring the
importance of vector startup overhead (=\nhalf/\rinf) in terms
of quantities known to the programmer (loop or vector length).
In the benchmark program, the two parameters are determined by a
least-squares fit of the data to the straight line defined by 
Eqn.(\ref{Eqn1}). A useful guide to the significance of \nhalf is to note
from Eqn.(\ref{Eqn2}) that 80 percent of the asymptotic performance is
achieved for vectors of length $4*\nhalf$. Generally speaking, \nhalf
values of upto about 50 are tolerable, whereas the performance
of computers with larger values of \nhalf is severely constrained
by the need to keep vector lengths significantly longer than \nhalf.
This requirement makes the computers difficult to program efficiently,
and often leads to disappointing performance, compared to the
asymptotic rate advertised by the manufacturer.  

RINF1 exists as part of the Hockney and Genesis benchmarks v2.0 and v2.1.1
~\cite{Hey91}. 
An independently written version forms module MOD1AC of the EuroBen
benchmarks ~\cite{StRi93}.

\subsection{Memory-Bottleneck Benchmarks: POLY1 and POLY2}

Even if the vector lengths are long enough to overcome the vector 
startup overhead, the peak rate of the arithmetic pipelines may not
be realised because of the delays associated with obtaining data
from the cache or main memory of the computer. The POLY1 benchmark attempts 
to quantify this dependence of computer performance on memory access
bottlenecks. The computational intensity, $f$, of a DO-loop is defined 
as the number of floating-point operations (flop) performed per memory 
reference (mref) to an element of a vector variable ~\cite{HoJe88}. 
The asymptotic performance, \rinf, of a computer is observed to increase 
as the computational intensity increases, because as this
becomes larger, the effects of memory access delays become negligible
compared to the time spent on arithmetic. This effect is characterised
by the two parameters (\rhat,\fhalf), where \rhat~ is the peak hardware
performance of the arithmetic pipeline, and \fhalf is the computational
intensity required to achieve half this rate. That is to say the
asymptotic performance is given by:
\begin{equation}
              \rinf = \frac{\rhat}{(1+\fhalf/f)}          \label{Eqn3}
\end{equation}                                        
If memory access and arithmetic are not overlapped, then \fhalf can
be shown to be the ratio of arithmetic speed (in Mflop/s) to memory
access speed (in Mword/s). The parameter \fhalf, like \nhalf, measures 
an unwanted overhead
and should be as small as possible. In order to vary $f$ and allow
the peak performance to be approached, we choose a kernel loop that
can be computed with maximum efficiency on any hardware. This is the
evaluation of a polynomial by Horner's rule, in which case the
computational intensity is the order of the polynomial, and both the
multiply and add pipelines are used. To measure \fhalf, the order of 
the polynomial is increased from one to ten, and the measured 
performance for long vectors is fitted to Eqn.(\ref{Eqn3}).

The POLY1 benchmark repeats the polynomial evaluation for each order
typically 1000 times for vector lengths upto 10,000, which would 
normally fit into the cache of a cache-based processor. Except for 
the first evaluation the data will therefore be found in the cache. 
POLY1 is therefore an {\em in-cache} test of the memory bottleneck 
between the arithmetic registers of the processor and its cache. 

POLY2, on the other hand, flushes the cache prior to each different
order and then performs only one polynomial evaluation,
for vector lengths from 10,000 upto 100,000, which would normally
exceed the cache size. Data will have to be brought from off-chip
memory, and POLY2 is an {\em out-of-cache} test of the memory
bottleneck between off-chip memory and the arithmetic registers.

The POLY1 benchmark exists as MOD1G of the EuroBen benchmarks 
~cite{StRi93}. POLY2 exists as part of the Hockney benchmarks.

\subsection{Arithmetic Benchmark Results}

As an indication of the type of results given by the proposed low-level
arithmetic benchmarks, Table-\ref{Table1} gives measurements made on a 
number of workstations, and microprocessor chips that are used as 
processing nodes in multiprocessor MIMD computers. 

\begin{table}
\centering
{\small
\parbox{5in}{\caption{\label{Table1}
Performance of some common numerical benchmarks on some common workstations 
and microprocessor chips used in MIMD computers. Measurements were made with 
the highest level of optimisation that ran, and are in Mflop/s for 64-bit 
precision, except where stated in parentheses. The units of \nhalf are vector 
length, and \fhalf are flop/mref (floating-point operations per memory 
reference). Results are for the best generally available compiler on the 
date shown. Those for the i860 are for the first Greenhills compiler which 
is known not to use many important i860 hardware features. 
Later more advanced compilers should give significantly better results.  
}}
\begin{tabular}{lccccccc}
\hline
               &Sun   &Solbourne&Stardent&Inmos&Intel&IBM RS/  &DEC   \\
 Benchmark     &Sparc1&System 5 &TS2025 &T800  &i860 &6000-530 &$\alpha$\\
               &      &         &       &20MHz &40MHz& 25MHz   &133MHz \\
\hline
      d/m/y &18/1/90&25/1/90 &8/8/89 &15/4/89&6/8/90&14/6/90&13/1/93  \\  
%           m/y & 1/90 &  1/90   &  8/89 &  4/89 & 8/90 & 6/90 & 1/93  \\  
\hline
   Linpackd    & 1.27 &  2.79   & 4.32  & 0.33  & 3.89 & 9.54 & 20.7  \\
     n=100 \\
\hline
   Livermore   & 2.36 &  4.64   &       & 0.72  & 8.76 & 31.8 & 46.6  \\
     Maximum \\
\hline
   Livermore   & 0.45 &  0.89   & 0.45  & 0.10  & 0.47 & 1.34 & 4.47  \\
     Minimum \\
\hline
   RINF1(32') \\
   \rinf      & 1.29 &  2.50   &19.29  &  0.34 & 4.62  & 5.13 & 33.8  \\
  (\nhalf)    &(0.30)& (1.00)  &(1.03) &   (0) &(3.61) &      &(12.2) \\
\hline
   POLY1 \\
   \rhat      & 2.50 &  5.18   &42.31  &        &10.59   &25.85 & 88.9\\
  (\fhalf)    &(0.77)& (0.60)  &(0.51) &        &(1.12)  &(0.34)& (0.71)\\
\hline
\hline
\end{tabular}
}
\end{table}

Table-\ref{Table1} shows that the DEC $\alpha$ chip outperforms
all other workstations and chips on all benchmarks by a significant 
margin, as befits the start of a new generation of chips. However, one 
cannot help being impressed by the figures. The remaining workstations 
and chips are compared with each other below.

Table-\ref{Table1} shows that the IBM RS/6000 chip set performs 
best on the LINPACKD100 benchmark, followed by the Stardent ST2025 
which has a vector architecture. 
The i860 performs significantly worse than the IBM 6000. However
the benchmark performance of both machines is expected to improve 
as their compilers develop.

Table-\ref{Table1} gives the maximum and minimum 
performance observed in the 24 Livermore loops. 
The minimum performance can be taken as giving the 
worst scalar arithmetic performance that is likely to be found, 
and the maximum gives the best performance that is likely to be 
seen on highly vectorisable loops. The computer with the best 
worst-performance, which is a very good metric to examine, is 
the IBM RS/6000 followed by the Solbourne. The best maximum 
performance is seen in the RS/6000 followed by the i860.

The RINF1 benchmark gives values of the \rnhalf parameters for the kernel 
A=B*C (vector = vector $\times$ vector), and shows the Stardent ST2025 
performing best with the
highest \rinf and lowest \nhalf, followed by the IBM RS/6000. 

The POLY1 benchmark shows 
the Stardent ST2025 with the highest peak performance, followed 
by the IBM RS/6000 and then the i860. Of these three, the value for 
the IBM is best, and the Stardent quite low, but the value greater 
than one for the i860 shows that there is a severe memory bottleneck 
problem with this chip that will prevent it from getting close to its
peak advertised performance on many problems.

\section{Multi-Processor Benchmarks}

\subsection{Communication Benchmarks: COMMS1 and COMMS2}

The purpose of the COMMS1, or {\em Pingpong}, benchmark \cite{Hoc88,Hoc91} 
is to measure the basic communication properties of a message-passing MIMD 
computer. A message of variable length, $n$, is sent from a master node to 
a slave node. The slave node receives the message into a Fortran data array, 
and immediately returns it to the master.  
Half the time for this `pingpong' exchange is recorded as the
time, $t$, to send a message of length, $n$.  
In the COMMS2 benchmark there is a message exchange in which two nodes 
simultaneously send messages to each other and return them. In this case
advantage can be taken of bidirectional links, and a greater bandwidth
obtained than is possible with COMMS1. 
In both benchmarks, the time as a function of
message length is fitted by least squares using the parameters
\rnhalf \cite{Hoc82,HoJe88} to the following linear timing model:
\begin{equation}
                     t = (n + \nhalf)/\rinf            \label{Eqn(4.1)}
\end{equation}
when the average communication rate is given by
\begin{equation}
    r = \frac {\rinf}{1+\nhalf/n} = \rinf \pipe (n/\nhalf) \label{Eqn(4.2)}
\end{equation}
\begin{equation}
\where \spten   \pipe (x) = \frac {1}{1 + 1/x}
\end{equation}
and the startup time is
\begin{equation}
                     t_0 = \nhalf/\rinf                \label{Eqn(4.3)}
\end{equation}
In the above equations, \rinf is the {\em asymptotic bandwidth} of 
communication
which is approached as the message length tends to infinity (hence the
subscript), and \nhalf is the message length required to achieve half
this asymptotic rate. Hence \nhalf is called the {\em half-performance
message length}. 

The importance of the parameter \nhalf is that it provides a yardstick with
which to measure message-length, and thereby enables one to distinquish 
the two regimes of short and long messages. 
For long messages $(n > \nhalf)$, the denominator in equation \ref{Eqn(4.2)}
is approximately unity and the average communication rate is approximately
constant at its asymptotic rate, \rinf 
\begin{equation}
                   r \approx \rinf                    \label{Eqn(4.3.5)}
\end{equation} 
For short messages $(n < \nhalf)$, the communication rate is best 
expressed in the algebraically equivalent form
\begin{equation}
                 r = \frac {\pi_0 n} {(1+ n/ \nhalf)}   \label{Eqn(4.4)}
\end{equation}
\begin{equation}
\where \spten     \pi_0 = t_0 ^{-1} = \rinf/\nhalf      \label{Eqn(4.5)}
\end{equation} 
For short messages, the denominator in equation \ref{Eqn(4.4)} will be 
approximately unity, so that
\begin{equation}
                 r \approx \pi_0 n = n / t_0            \label{Eqn(4.6)}
\end{equation}
In sharp contrast to the approximately constant rate in the long-message 
limit, the communication rate in the short message limit is seen to be 
approximately proportional to the message length. 
The constant of proportionality, $\pi_0$, is known
as the {\em specific performance}, and can be expressed conveniently
in units of kilobyte per second per byte (kB/s)/B or k/s. 

Thus, in general, we may say that \rinf characterises the long-message 
performance and $\pi_0$ the short-message performance. The COMMS1 benchmark 
computes all four of the above parameters, 
$(\rinf, \nhalf, t_0, \rmand \pi_0)$,
because each emphasises a different aspect of performance. However only 
two of them are independent. In the case that there are different modes
of transmission for messages shorter or longer than a certain length, the
benchmark can read in this breakpoint and perform a separate least-squares 
fit for the two regions. An example is the Intel iPSC/860 which has a different
message protocol for messages shorter than and longer than 100 byte. 

Because of the finite (and often large) value 
of $t_0$, the above is a {\em two-parameter} description of communication 
performance. It is therefore incorrect, and sometimes positively misleading,
to quote only one of the parameters (e.g. just \rinf, as is often done) to
describe the performance. The most useful pairs of parameters are \rnhalf
$(\pi_0,\nhalf)$ and $(t_0,\rinf)$, depending on whether one is concerned
with long vectors, short vectors or a direct comparison with hardware times. 
Note also that, although \nhalf is defined
as the  message length required to obtain half the asymptotic rate \rinf,
the two parameters \rnhalf are sufficient to calculate the average rate for
any message length via equation \ref{Eqn(4.2)}, or equivalently using 
$\pi_0$ instead of \rinf via \ref{Eqn(4.4)}.

The COMMS1 and COMMS2 benchmarks exist as part if the Genesis benchmarks
v2.0 and 2.2.1 ~\cite{Hey91}.

\subsection{Example Results for the COMMS1 benchmark}

We report below results for the COMMS1 benchmark on the \Suprenum,
and Intel iPSC/860 ~\cite{Hoc91} and Touchstone Delta ~\cite{HoCa92} 
computers. 

Table-\ref{Table4.1} gives the values obtained for the communication 
parameters, in the version of the benchmark using the native \Suprenum
extensions to the Fortran90 language. These include a SEND and RECEIVE
language statement with a syntax similar to that of the Fortran READ
and WRITE statement.
The asymptotic stream rate, or bandwidth, (\rinf) shows considerable
variation on the Suprenum, depending on how the data to be transferred
is specified in the I/O list of the SEND statement.  A variable length
array in Fortran90 syntax in single precision achieves 0.67 MB/s,
whereas the same statement specified in double precision achieves 4.8
MB/s.   This double-precision rate is about twice that observed
on the iPSC/860 with their CSEND Fortran subroutine, which sends an array 
whose length is specified in bytes. The principal difference between the 
two computers is the magnitude of the startup time, $t_0$, which is 
$73\mu s$ on the iPSC/860 compared with about 3ms on the Suprenum.  Since 
the startup time, via $\pi_0$, determines the transfer rate for short 
messages (say $<100$B), we see that the Suprenum is 45 times slower than 
the iPSC/860 for short messages. On the other hand the Suprenum has almost 
twice the stream rate for long messages (as seen by the value of \rinf), 
provided the most favourable format (i.e. double precision or 64-bit)
is used in the I/O list.  One may compute from these numbers that the
iPSC/860 is faster at transferring messages for all message lengths
less than 16,481 Byte. The longer startup time on Suprenum results in 
larger values of \nhalf, showing that longer messages are needed to 
achieve any given fraction of the asymptotic rate.

The results for the Touchstone Delta show that this computer has the
fastest short and long message performance, judged respectively by the
values of $\pi_0$ and \rinf. However the improvement of short message
performance over the iPSC/860 is only marginal, and the long message 
performance is only about one quarter of the advertised bandwidth of 
25MB/s. However harware and software improvements made since the 
measurements were made should have improved the results.

\begin{table}
\centering
{\small
\parbox{3.5in}{
\caption{\label{Table4.1}
          Values of (\rinf,\nhalf) for the communication of messages
          between two nodes of the same cluster on the Suprenum and 
          neighbouring nodes on the Intel iPSC/860 and Touchstone
          Delta computers. The Delta measurements were made on 17 Jan.
          1992, and should have been improved by subsequent hardware
          and software changes.}}
\begin{tabular}{llcccc}
\hline
Specification   & Range      & \rinf   & \nhalf   & $t_0$  & $\pi_0$   \\
                & B*         & MB/s    & B        & ms     & k/s       \\
\hline
   SUPRENUM                                                            \\
sp SEND A(1:N)  &            & 0.67    &  2041    & 3.05   & 0.328     \\
dp SEND A(1:N)  &            & 4.82    & 12740    & 2.64   & 0.378     \\
\hline
   INTEL iPSC/860                                                      \\
CSEND (,A,N,,)  & $N<100$    & 2.36    & 179      & 0.074  & 13.5      \\
                & $N>100$    & 2.80    & 560      & 0.200  &  5.0      \\
\hline
   INTEL Delta                                                         \\
CSEND (,A,N,,)  & $N<512$    & 3.48    & 213      & 0.061  & 16.3      \\
                & $N>512$    & 6.76    & 892      & 0.132  & 7.57      \\
\hline
* B - byte

\end{tabular}
}
\end{table}

% ----------------------------------------------------------------------------

\subsection{Total Saturation Bandwidth: COMMS3}

To complement the above communication benchmarks, there is a need for
a benchmark to measure the total saturation bandwidth of the complete
communication system, and to see how this scales with the number
of processors. A natural generalisation of the COMMS2 benchmark
could be made as follows, and be called the COMMS3 benchmark:
Each processor of a $p$-processor system send a message of length $n$
to the other $(p-1)$ processors. Each processor then waits to receive
the $(p-1)$ messages directed at it. The timing of this generalised
'pingping' ends when all messages have been sucessfully received by all
processors; although the process will be repeated many times to obtain
an accurate measurement, and the overall time will be divided by the
number of repeats. The time for the generalised pingping is the time to
send $p(p-1)$ messages of length $n$ and can be analysed in the same way
as COMMS1 and COMMS2 into values of \rnhalf. The value obtained for \rinf
is the required total saturation bandwidth, and we are interested in how
this scales up as the number of processors $p$ increases and with it the
number of available links in the system.

This benchmark does not exist, but Roger Hockney will develop a trial
version for the Intel iPSC, followed by PARMACS and PVM. Perhaps suitable 
and better benchmarks exist elsewhere. Please send in your suggestions.

\subsection{Communication Bottleneck: POLY3}

POLY3 assesses the severity of the communication bottleneck.
It is the same as the POLY1 benchmark except that the data for the
polynomial evaluation is stored on a neighbouring processor. The value of
\fhalf obtained therefore measures the ratio of arithmetic to communication
performance. Equation ~\ref{Eqn3} shows that the computational intensity 
of the calculation must be significantly greater than \fhalf (say 4 times
greater) if communication is not to be a bottleneck. In this case
the computational intensity is the ratio of arithmetic performed on a
processor to words tranferred to/from it over communication links.
In the common case that the amount of arithmetic is proportional to
the volume of a region, and the data communicated is proportional to
the surface of the region, the computational intensity is increased as
the size of the region (or granularity of the decomposition) is increased. 
Then the \fhalf obtained from this benchmark is directly related 
to the granularity that is required to make communication time unimportant.

The POLY3 benchmark does not exist, although native versions have been
used on transputer systems ~\cite{Hoc91}. A trial benchmark will be 
prepared for Intel iPSC computers by Roger Hockney, followed by PARMACS 
and PVM versions.

\subsection{Synchronisation Benchmarks: SYNCH1}

SYNCH1 measures the time to execute a barrier synchronisation statement
as a function of the number of processes taking part in the barrier.
The practicability of massively parallel computation with thousands
or tens of thousands of processors depends on this barrier time not
increasing too fast with the number of processors. The results are quoted
both as a barrier time, and as the number of barrier statements
executed per second (barr/s).

The SYNCH1 benchmark exists as part of Genesis v2.0 and v2.1.1 ~\cite{Hey91}.

\section{Summary of Benchmark Status}

The following Table-\ref{Table3} summarises the current state of the 
proposed low-level benchmarks, and the properties they are intended
to measure.

\begin{table}
\centering
{\small
\parbox{5in}{
\caption{\label{Table3}
          Status of proposed Low-Level benchmarks. Note we abbreviate
          performance (perf.), arithmetic (arith.), communication (comms.),
          operations (ops.).}}
\begin{tabular}{llcccc}
\hline
Benchmark       & Measures             & Parameters & Exists &  Author     \\
                &                      &            &        &             \\
\hline
SINGLE-PROCESSOR                                                           \\
   TICK1        & Timer resolution     & tick interval & Genesis & Hockney \\
   TICK2        & Timer value          & wall-clock check& Genesis & Hockney \\
   LINPACKD100  & typical arith. perf. & job Mflop/s  & Linpack & Dongarra  \\
   LIVERMORE    & Arith. perf. range   & loop Mflop/s & Livermore & McMahon \\
   RINF1        & Basic Arith. ops.    & \rnhalf    & Genesis & Hockney \\
   POLY1        & Cache-bottleneck     & \rfhalf    & EuroBen & Hockney \\
   POLY2        & Memory-bottleneck    & \rfhalf    & Hockney & Hockney \\
\hline
MULTI-PROCESSOR                                                         \\
   COMMS1       & Basic Message perf.  & \rnhalf    & Genesis & Hockney \\
   COMMS2       & Message exch. perf.  & \rnhalf    & Genesis & Hockney \\
   COMMS3       & Saturation Bandwidth & \rnhalf    & No      & Hockney \\
   POLY3        & Comms. Bottleneck    & \rfhalf    & No      & Hockney \\
\hline

\end{tabular}
}
\end{table}

% ----------------------------------------------------------------------------