Subject: Compact applications

From: [email-masked] (David Walker)
Date: Fri, 21 May 1993 08:42:55 -0400

Dear Compact Applications People,

At last I have roughed out some notes on compact applications to serve as
a discussion for next weeks meeting in Knoxville. See you there,

David
------------------ Latex file below --------------------------------
%file: compac2.tex
\chapter{Compact Applications}
\footnote{assembled by David Walker for Compact Applications subcommittee}

\section{Introduction}
\label{sec:compact.intro}
While kernel applications, such as those described in Chapter 4, provide
a fairly straightforward way of assessing the performance the parallel
systems they are not representative of scientific applications in general
since they do not reflect certain types of system behavior. In particular,
many scientific applications involve data movement between phases of
an application, and may also require significant amounts of I/O. These types
of behavior are difficult to gauge using kernel applications. 

One factor
that has hindered the use of full application codes for benchmarking parallel
computers in the past is that such codes are difficult to parallelize and to
port between target architectures. In addition, full application codes that
have been successfully parallelized are often proprietary, and/or subject
to distribution restrictions. To minimize the negative impact of these factors
we propose to make use of compact applications in our benchmarking effort.

Compact applications are typical of those found in research environments 
(as opposed to production or engineering environments), and usually consist of 
up to a few thousand lines of source code. Compact applications are distinct 
from kernel applications since they are capable of producing scientifically
useful results. In many cases, compact applications are made up of several
kernels, interspersed with data movements and I/O operations between the 
kernels.

In this chapter we will discuss a number of compact applications in terms of 
their purpose, the algorithms used, the types of data movements required, 
the memory requirements, and
the amount of I/O. The compact application below are not meant to form a 
definite or complete list.

\section{Proposed Compact Application Benchmarks}
\label{sec:compact.proposed}
To ensure that those areas of scientific computing that make the most use of
high performance computers are adequately represented in the benchmark
suite we shall classify compact applications by scientific field.

\subsection{Plasma Physics}
\label{subsec:plasmas}
Plasma physics is a large consumer of high performance computer cycles. Among
the areas studied are the design of tokamaks, high power microwave devices, and 
astrophysical plasmas. It would be nice to have a compact application from 
each of these three fields in the benchmark suite. Currently we have Hockney's
device simulation, LPM1, from the GENESIS suite.

\subsubsection{Electronic Device Simulation with LMP1}
\label{subsubsec:lpm1}
LMP1 is a time dependent simulation of an electronic device
using a particle-mesh or PIC-type algorithm. It uses a two-dimensional
$(r,z)$ geometry with the fields being computed on a regular mesh
of size $33\times 75\cdot\alpha$, where $alpha$ is a size parameter that can
take the value 1, 2, 4, and 8, corresponding to runs with between about 700 and
6000 particles.

\subsection{Quantum Chromodynamics}
\label{subsubsec:qcd}
Quantum Chromodynamics (QCD) is the gauge theory of the strong
interaction which binds quarks and gluons into hadrons, which make up the
constituents of nuclear matter. Analytical perturbation methods can be applied
to QCD only at high energies, hence computer simulations are necessary to study
QCD at lower, more realistic, energies. In these lattice gauge theory
simulations the quantum field is discretized onto a periodic, four-dimensional,
space-time lattice. Quarks are located at the lattice sites, and the gluons
that bind them are associated with the lattice links. The gluons are
represented by SU(3) matrices, which are a particular type of $3\!\times\! 3$
complex matrix. A major component of the QCD code involves updating these
matrices.

\subsubsection{Quenched QCD}
\label{subsubsec:quenched}
The QCD code in the Perfect benchmark suite is derived from the work of
Fox, Flower, Otto, and Stolorz at Caltech. The Perfect QCD code uses the 
Cabbibo-Marinari pseudo heat bath algorithm to update the SU(3) matrices on
the lattice links. This algorithm uses a Monte Carlo technique to generate a 
chain of configurations which are distributed with a probability proportional
to $\exp{(-S(U))}$, where $S(U)$ is the action of the configuration $U$.
If the only contributions to the action come from the gauge field then
the action is local. The inclusion of dynamical fermions gives rise to a
nonlocal action. This code ignores the effects of dynamical fermions, and so
represents a pure-gauge model in the quenched approximation.

A major component of this QCD code is the updating of the SU(3) matrices
associated with each link in the lattice, and it is this operation which
is benchmarked in the Perfect timings. Two basic operations are involved in
updating the lattice. The first is the multiplication of SU(3) matrices,
and the second is the generation of pseudo-random numbers.

\subsubsection{Genesis QCD}
\label{subsubsection:dynamical}
Is the Genesis benchmark QCD1 similar to the Caltech QCD code. Which one
should be used?

\subsection{General Relativity}
\label{subsec:gr}
\subsubsection{Evolution of Gravitational Field}
The Genesis code GR1 solves a system of hyperbolic PDEs, derived from general
relativity which describe the evolution of a gravitational field from an
initial state. Although conceptually similar to the solution of the wave
equation the equations are long and complicated. This application solves the
axisymmetric problem to reduce the problem to manageable size. Solution of
the general problem requires three orders of magnitude more compute power,
and is likely to become of substantial interest as more powerful parallel
machines are developed.

\subsubsection{Quantum Theory of Gravity}
\label{subsec:gravity}
This code, which derives from the work of Sorkin and Daughton of
Syracuse University, is part of an effort to provide a
satisfactory quantum theory of gravity by the use of causal set
theory$\ldots$whatever that is. The main computational task is the LU
factorization of large, dense matrices ($10000\times 10000$).

\subsection{Climate and Weather Prediction}
\label{subsec:climate}
Mesoscale weather prediction and global climate modeling have become
important application areas in recent years. They typically involve the
solution of nonlinear PDEs.

\subsubsection{Spectral Solver for the Shallow Water Equations}
\label{subsubsec:swe}
The spectral transform method 
is the standard numerical technique
used to solve partial differential equations on the sphere in
global climate modeling. For example, it is used in CCM1 
(the Community Climate Model 1), and its successor CCM2.
The solution of the shallow water equations on a sphere constitutes an 
important component in such global climate models.
The SSWMSB code uses the spectral transform method to solve the shallow water
equations on the surface of a sphere which is discretized as a regular
longitude-latitude grid. In each timestep the state variables of 
the problem are transformed
between the physical domain, where most of the physical forces are calculated,
and the spectral domain, where the terms of the differential equation
are evaluated. This transformation involves first the evaluation of FFTs along
lines of constant latitude, followed by Legendre integration (i.e., weighted
summation) over longitude.

\subsubsection{Helmholtz Solvers for Meteorological Modeling}
\label{subsubsec:helmholtz}
The Genesis suite includes two meteorological applications based on 
Helmholtz solvers. One uses a pseudo-spectral solution method, and the other
a multigrid algorithm.

\subsection{Molecular Dynamics}
\label{subsec:moldyn}

\subsubsection{Dislocation Studies in Crystals}
\label{subsubsec:dislocation}
In parallel Fortran 77 plus message passing code has been developed at ORNL to 
study dislocation phenomena in crystals. This three-dimensional code divides
space into cells, with each processor being assigned a rectangular block of
cells. Each cell contains a set of particles. Communication is necessary to
exchange particles lying in cells on the boundary of a processor with a
neighboring processor. Particles must also be migrated between processors
as they move in space.

\subsubsection{The Genesis Molecular Dynamics Code}
\label{subsubsec:genesis_md}
I don't know much about this, but I expect it's similar to the ORNL code.

\subsubsection{The PERFECT Molecular Dynamics Code}
\label{subsubsec:perfect_md}
The Perfect benchmark suite included two molecular dynamics code, both of
which use data sets that are too small to be used to evaluate current
parallel computers. BDNA which simulates the hydration structure of potassium
counterions and water in a B-DNA molecule, involves 1500 water molecules and
20 counterions. MDG performs a molecular dynamics calculation on 343 water
molecules in the liquid state.

\subsection{Geophysics}
Two important geophysics computations are flow through porous media and
seismic migration. The Perfect suite includes a seismic migration code,
MG3D. This code is dominated by FFTs. A parallel code for modeling groundwater
flow is under development at ORNL and may be a good code to include in the
suite as an example of a flow through porous media code.

\subsection{Other Codes}
Clearly we would want to include CFD codes, astrophysics codes such as the
tree-based simulations of gravitating systems, quantum chemistry and
superconductor simulations. We also need to include codes from the NAS, NPAC,
PERFECT2, and SLALOM benchmark suites, as well as providing better 
descriptions of the codes above.

\section{Concluding Remarks}
There are probably two or three dozen compact applications that
we might consider for inclusion in the benchmark suite. We should consider
what is a reasonable number of codes to include, and the criteria for
accepting a code in terms of documentation, usefulness, and software quality.

---

From: "Michael W. Berry" <[email-masked]>
Date: Fri, 21 May 1993 09:06:39 -0400

Fellow Compact Applic. Members: Here is a copy of the minutes
from the SPEC/Perfect meeting I attended in Hunstville.  Some
of this information may be useful to PBWG.

Mike B.
---------------------------------------------------------------

                        Draft Minutes: The SPEC Perfect Group
                                   11-13 May 1993

          The Perfect Club Steering Committee voted to merge with the  SPEC
          organization.  The first joint meeting with SPEC occurred  during
          11-13 May 1993.  The original SPEC organization has been modified
          so that  the name  "SPEC" refers  to the  non-profit  corporation
          which acts as  a financial umbrella  for benchmarking  subgroups.
          The original SPEC  group is now  known as the  SPEC Open  Systems
          Group.  The Perfect Club is now known as the SPEC Perfect Group.

          In accordance with the  vote taken by  David Schneider in  April,
          the initial  SPEC Perfect  Steering Committee  includes  Margaret
          Simmons (LANL), George Cybenko(Darmouth), David Schneider (CSRD),
          John Larson (CSRD),  Mike Berry (U.of  Tenn), Satish Rege  (DEC),
          Joanne Martin (IBM), and Philip Tannenbaum (HNSX).  This  meeting
          was attended by David Schneider  (CSRD), Mike Berry (U.of  Tenn),
          Satish Rege  (DEC),  Philip  Tannenbaum  (HNSX),  Leo  Boelhouwer
          (IBM-Kingston,  representing   Joanne   Martin),   Jacob   Thomas
          (IBM-Austin), Larry Gray  (Chairman, SPEC BOD),  and Rod  Skinner
          (Treasurer, SPEC).   Hwa Lai (Fujitsu)  attended as an  observer.
          Various SPEC Open  Systems members  periodically sat  in.   David
          Schneider indicated  that  he  anticipated  Cray  Research  would
          rejoin because of marketing necessity.

          The meeting  began  with David  Schneider,  Larry Gray,  and  Rod
          Skinner presenting the framework for  the merger.  The SPEC  Open
          Systems Group  and  the SPEC  Perfect  Group will  be  autonomous
          subgroups within  SPEC.   SPEC  itself  will act  as  a  business
          umbrella organization.  Each Group will assess dues and  allocate
          budgets independently.   The  overhead which  SPEC Perfect  Group
          will  be  responsible  for   will  include  legal  retainer   and
          accounting fees  for  NCGA,  and additional  costs  of  printing,
          duplication,  distribution,  or  other  services  that  the  SPEC
          Perfect Group may elect  to utilize in the  future.  It was  also
          stated that the  SPEC organization was  flexible on many  issues,
          but the  underlying  requirement  was to  ensure  that  corporate
          non-profit  status  regulations  are  not  violated.    SPEC   is
          incorporated as a non-profit organization in California.

          It was  generally  agreed  by  all that  mutual  trust  would  be
          required from SPEC Open Systems  Group and SPEC Perfect Group  to
          minimize formality and unnecessary bureaucracy.

          The Perfect Group will be given one SPEC BOD seat on a  temporary
          basis until January 1994.  The  SPEC BOD currently consists of  5
          members that  includes HP,  Intel, Sun,  ATT/NCR, and  IBM.   The
          Perfect Group seat will add 1 member to the BOD.  In January 1994
          this 6th BOD  seat will  be open for  voting by  the entire  SPEC
          membership (SPEC Perfect Group and  SPEC Open Systems Group).   A
          discussion about who should fill the temporary SPEC Perfect Group
          BOD seat resulted in agreement  that University people could  not
          practically take the  position because  of travel  expense.   IBM
          already was  represented  on the  SPEC  BOD, so  David  Schneider


          nominated Satish  Rege  (DEC)  and Philip  Tannenbaum  (HNSX)  as
          candidates for  the  BOD  seat.    Leo  Boelhouwer  seconded  the
          nomination  for  Philip  Tannenbaum;  Mike  Berry  seconded   the
          nomination for Satish Rege.   A vote will  be conducted by  email
          on/about 1 June 1993.  The initial  7 Steering Committee  members
          are the eligible voters.

          During June  a  press  announcement about  the  merger  would  be
          jointly written.

          There was discussion about  inclusion of academic and  government
          members.   As  a  result of  SPEC  non-profit  requirements,  all
          members must be  either full members  ($5,000/year) or  associate
          members ($1,000/year).    It was  agreed  that few  academics  or
          government members could  acquire funding for  membership.   SPEC
          Perfect Group  Steering  Committee  could elect  to  sponsor  the
          memberships of  selected  individuals;  and  certain  individuals
          could  be  included  by  creation  of  "SPEC  Fellows"  or  "SPEC
          Affiliates" whereby  specific services  could  be paid  for  with
          membership.     Seeking  industrial   sponsorship  for   academic
          participation was  discussed as  desireable.   Each  member  will
          initiate a "check is  in the mail"  process for their  membership
          fees.   Diane  Dean,  NCGA,  2722  Merrilee  Drive,  Fairfax,  VA
          22301-4499 (703-698-9600  x318) is  our contact  in this  regard.
          SPEC Open  Systems  Group  members  received  6  free  pages  for
          SPEC/OSG reporting  in the  publications; additional  pages  were
          billed at $500  each--it was  noted that DEC  purchased 60  extra
          pages in the last publication to kick off a new product line.

          The SPEC Perfect group organization was discussed.  It was agreed
          that the SPEC Perfect Group should have a Chairman, a  Secretary,
          and a Technical Coordinator.   The Chairman would be  responsible
          for interfacing  with  SPEC  and the  SPEC  Open  Systems  Group,
          organizing meetings, and general management.  The Secretary would
          be   responsible    for   generating    minutes   and    handling
          correspondence.  The Technical  Coordinator would be  responsible
          for benchmarking status,  benchmark production and  distribution,
          coordinating the benchmark subgroups,  and being the focal  point
          for technical issues.  Each benchmark subgroup would have its own
          leadership.

          Temporary assignments were accepted to fill these positions until
          the next SPEC Perfect Group  meeting, targeted for August at  ATT
          (Chicago).    Rege  Satish  is  the  temporary  Chiarman,  Philip
          Tannenbaum  the  temporary  Secretary,  and  Leo  Boelhouwer  the
          temporary Technical Coordinator.   Specific action items for  the
          period include:

             Completing the benchmark codes
             Generating verification tests and timing instrumentation
             Publishing minutes
             Writing a  solicitation for  vendors  and industry  to  attract
             membership or sponsorship support


          A discussion about the benchmark rules and reporting resulted  in
          general  agreement  that  there  would  be  baseline  ("As   Is")
          executions which  allowed only  the minimal  changes required  to
          obtain correct  results.   There would  also be  an optimized  or
          alternative solution execution which would allow unlimited use of
          standard vendor libraries and unlimited rewriting in a high level
          language.  

          It was agreed  that the benchmark  programs would be  distributed
          via netlib  or  anonymous ftp.    Text  would be  added  to  each
          benchmark program  requiring that  any use  of benchmark  results
          from the program, which are  not formally accepted and  published
          by SPEC  Perfect  Group,  must   state  "these  results  are  not
          officially approved  and  reported  by  the  SPEC  Perfect  Group
          Steering Committee.    They may  not  be directly  comparable  to
          accepted and verified results."
          Only actual execution results would be permitted.  All executions
          must be  on  hardware  and  software  systems  that  are  current
          products or  which  will be  generally  available in  the  market
          within 6 months.  

          There was  a  spirited debate  on  the  metrics to  be  used  for
          reporting results.  Discussion about  the pros and cons of  using
          normalized  ratings,  MFLOPS,  wall  clock  times,  and  absolute
          numbers took  place. The  discussion  resulted in  the  benchmark
          publications including 1)elapsed wall clock time, 2)startup time,
          3)time step  timing,  3)cleanup  time,   4)total  user  cpu  time
          accumulated, and 5)total system cpu time accumulated per program.
            No MFLOPS rate will reported.   This was agreed to be the  most
          scientifically  sound  approach  that  would  be  meaningful  and
          unambiguous.

          All execution results presented for approval and publication must
          include  sufficient   detail  of   the  hardware   and   software
          configuration such that the  run could be essentially  duplicated
          with comparable  timings.   Acceptable  results will  have  valid
          answers and meet  SPEC Perfect Group  standards for code  changes
          and execution requirements.   Optimized and alternative  solution
          results must include the entire  program code as executed, and  a
          statement that the code  may be used,  without restriction, as  a
          SPEC Perfect Group baseline benchmark  code.  All vendor  library
          codes  used   must  include   copies  of   the  relevant   vendor
          documentation page that include sufficient detail to describe the
          processes done within  the library routine.   New vendor  library
          routines   must   have    copies   of   equivalent    preliminary
          documentation.   All  library  routines used  must  be  generally
          available to all vendor customers, and must either be  documented
          products, or  become  documented  products  within  6  months  of
          benchmark submission.    Results on  prototype  or  preproduction
          systems could  be removed  from  publication if  the  benchmarked
          products were not released within the 6 month window.

          The goal  is to  provide  all codes  in  a FORTRAN77  version,  a
          FORTRAN90 version, and a message passing version.  It was  agreed


          that version control  should be  instituted so  that all  results
          would be grouped according to benchmark version.  If any one code
          in a  benchmark group  changed,  all codes  would receive  a  new
          version number.  The benchmark groups will be aligned to  address
          vertical industrial areas such as petroleum, chemistry,  finance,
          etc. 

          The codes available  for the initial  release include the  FDMOD,
          FKMIG, and  SEIS from  the ARCO  suite, QCD,  FALSE, PUEBLO,  and
          TURB3D.   The ARCO suite codes are farthest along.  All codes are
          expected  to  represent  scalable  problem  solutions  that   are
          appropriate to vector, vector parallel, and MPP architectures.  A
          goal is to  maintain the benchmark  set at a  level whereby  only
          supercomputer class  and extreme  high end  workstations/clusters
          could reasonably  execute the  problems.   There is  no  specific
          exclusion intended; this goal was stated in order to maintain the
          SPEC Perfect Group focus on  true supercomputing rather than  the
          broader high performance computing classification.  The goals may
          not all be addressed initially because of pratical limitations in
          how much can be accomplished with available resources.

          Coding and  language standards  were discussed.   Proposals  were
          made.  John Larson''s work in this area will be circulated.   Leo
          Boelhouwer will  edit  the  V1 execution  rules  and  present  an
          updated draft for approval during the next meeting.  

          Language standards  were  presented as  a  basis for  creating  a
          benchmark code  standard  by  David  Schneider.    They  included
          numerous items that were accepted by the group, and a few  (noted
          below) where no final conclusion was made.

               Variables could not exceed 31 characters
               No Pointers
               No DOUBLE PRECISION; REAL*8 and COMPLEX*16 should be used
               No CHARACTER-Floating Point equivalences
               No Hollerith constants or data
               No 128 bit requirements (REAL*16, COMPLEX*32)
               All 64 bit constants should be specified in D format
               All 32 bit constants should be specified in E format
               Machine constant limitations were discussed--no  conclusions
          agreed
               INTEGER*8 and LOGICAL*8 should not be used unless  necessary
          for execution
               Tests  for  floating   point  equality  were   discussed--no
          conclusions agreed


               Known vector directive information  will be translated to  a
          "C*PERFECT" syntax to
                    preserve information; it will be explicitly  prohibited
          from implementing compiler
                    recognition of "C*PERFECT" information.
               DO WHILE and DO-ENDDO syntax is allowed
               "!" inlined comments were discussed--no conclusions agreed

          Additional action items were summarized:

               Distribute old by-laws for review (DS)
               Review old by-laws and offer suggestions for revision (all)
               Contact NCGA regarding our new status (DS)
               Present our proposals for membership specific issues to  the
          SPEC BOD (SR)
               Identify manpower  requirements  to  complete  V2  benchmakr
          suite (all)
               Transfer "Perfect Benchmark" trademark  from U.Ill. to  SPEC
          (DS)
               Distribute Minutes (PT)
               Set up address and email lists (DS)
               Next meeting  at ATT,  Chicago, in  August (with  SPEC  Open
          Systems Group) (all)
               Schedule a benchathon to finalize all V2 inital codes (all).

---
Michael W. Berry     ___-___  o==o======   .   .   .   .   .
Ayres 114         =========== ||//         
Department of             \ \ |//__        
Computer Science          #_______/        [email-masked]
University of Tennessee                    (615) 974-3838 [OFF]
Knoxville, TN 37996-1301                   (615) 974-4404 [FAX]