Subject: Minutes

From: "Michael W. Berry" <[email-masked]>
Date: Thu, 30 Mar 1995 08:38:29 -0500

Here are the minutes I recorded from the most recent Parkbench
meeting.  Please review, correct, and e-mail your changes.
Please pay particular attention to *** ACTION ITEM ***'s.
Thanks,
Mike

-----------------------------------------------------------------
Minutes  of Parkbench Meeting - Knoxville Hilton - March 27, 1995
-----------------------------------------------------------------

Attendee List:

           David Bailey    NASA                  [email-masked]
           Mark Baker      Univ. of Southampton  [email-masked]
           Michael Berry   Univ. of Tennessee    [email-masked]
           Jack Dongarra   Univ. of Tenn./ORNL   [email-masked]
           Tony Hey        Univ. of Southampton  [email-masked]
           Kenneth Koch    Los Alamos Nat'l Lab  [email-masked]
           Mike Kolatis    Univ. of Tennessee    [email-masked]
           David MacKay    Intel SSD             [email-masked]
           Fiona Sim       IBM (Kingston)        [email-masked]
           David Snelling  Univ. of Manchester   [email-masked]
           David Walker    Oak Ride Nat'l Lab    [email-masked]

At 9:30am,  Tony H. opened the meeting and asked if there were any
corrections (some spelling errors).   Tony indicated that there should
be a formal release of the codes.  The status of the WWW Repository
was reviewed.  The codes are not really usable in the current form.
Reorganization of the web pages is needed.  Jack D. agreed that the
web pages simply need to be redone.   There are collections that have
nothing to do with the suite.  Fiona S. indicated that it was very
confusing when trying to download the suite.   There has been some
duplication of work. 

Tony H. then outlined "Proposed Parkbench Activities":

1. Benchmark Consolidation

   Release 1.0 of the suite is to be fully tested by June 1, 1995.
   Items: low-level , kernels, compact applic, HPF compiler benchmarks.

    Release 1.0
    ----------

    Low level - need common makefiles and I/O (small amount of work needed);
    there are some typo errors in the makefiles; all in Fortran and can work
    with PVM 3.3.7 version; there are 7-8 codes in total.
    Genesis V. 3 ties them together (M. Kolatis had worked with older version).
    Via email, Roger Hockney suggested that UT and Southampton (Ian
    Glendinning) should coordinate to finalize the low-level kernels.  Roger
    H. stressed that as many measurements as possible be collected for the
    database and that duplication of results be constantly checked.  Roger H. 
    indicated that he is working on a "graphical profile of performance" which
    will be based on scatter plots of all (rinf,nhalf) values on the
    (nhalf,rinf)-plane.  Roger H. encourages all Parkbench members to submit
    their low-level measurements to the Southampton graphical data base (GBIS).

    Kernels -  NAS kernels : MG,CG,EP,IS,FFT(3-D);   these are vanilla versions.
    Matrix kernels : MM, TP, LU, QR, TRD;  TP needs some work and
    the others need to be standalone and not dependent on BLACS.
    User will just need to provide the communication layer.  Need
    to have "load and go" versions.  A new version just installed
    but  BLACS are not included (but will be added with tar file).
    PVM+BLAS needed by vendors for these codes. 

    Work needed for kernels: common timer, common header file, common makefiles,
    common I/O (input,includes), standard documentation of parameters,
    standard problem sizes, public domain PVM -vs- vendor-based PVM,
    language conformance, NAS pencil+paper -vs- PARKBENCH versions,
    verification of results.  Ken K. suggested that a list of PVM files
    needed be specified.  He asked if one should look at process control
    calls specifically?  Codes are not exactly Fortran-77.   Need to point
    to some well-defined extension (DOD sanctioned Mill-Spec extension).
    Don't want Fortran-90 infiltration right now.  David B. suggested that
    someone should check for use of "SAVE" option.  A preprocessor-based
    implementation is not desired at this time (suggested by Tony H.)
    Need to have a reporting mechanism - ask vendors for a general reporting
    strategy.   Will have 2 sets of results for NAS kernels - vendors have
    their own versions and how to distinguish the results?  Optimized
    libraries can be used with vanilla version (compiler options also).
    Mark B. suggested that there are really 3 versions: true vanilla (no flags),
    code reordering+libraries, vendor-best approach.  David B. suggested
    that there only be 2 levels : level 1 (apply any compiler options but
    applied to all routines) and level 2 (do anything to it).   Jack D. wants
    same compiler options for entire suite - not just w/in 1 code.  SPEC
    has a set of rules for compiler options.

                        *** ACTION ITEM ***
    Fiona S. will provide SPEC compiler option rules for review.

    David B. indicated that simple checks of correctness provided in NAS. 
    Genesis codes are not self-checking.
    David S. stressed that self-checking is preferred (correctness issue).

    Tony H. proposed to drop PDE1 due to duplication (R/B ordering); a 1-D FFT 
    could be included and should a pencil+paper I/O benchmark be included.  
    David B. suggested to hold off on the FFT for now.   David S. agreed.  A
    general discussion of how to formalize an I/O benchmark followed.  David B.
    and Michael B. suggested that the compact applications could provide I/O 
    benchmarking activities within them. For Release 1, it should be avoided. 
    Jack D. mentioned an activity by Paul Messina at Caltech to address I/O 
    benchmarking.   Jack D. proposed 30 megabytes or 1000 order matrix (for 
    small problem).  Medium size is 3 gigabytes (order of 10,000 matrix), and
    large problem size would be 30 gigabytes (30,000 order matrix).   Fiona S.
    indicated IBM typically ties problem size to number of procs.
    David B. indicated that there is a NAS Parallel Benchmarks 2 effort in
    existence.  Use 2 problem sizes for NAS parallel benchmarks and 3 problem 
    sizes for matrix kernels for now. 

                        *** ACTION ITEM ***
    David B. will investigate a 3rd problem size for NAS Parallel Kernels. 

     Compact Applications -  (discussion began at 10:50am).

     NAS CFD codes (need PVM versions);  David B. indicated that MPI versions
     are being developed.

     ARCO (1.1 version) - by Mosher (Fiona S. indicated that "Resource 2000"
     effort presents problem with multiple versions with different I/O).

     Enviro. Modeling - POLMP,  PSTSWM (PICL-based) 

     QCD - SOLVER

     GAMESS (a public domain PVM version possible?);  SPEC has the US version
     (2 versions one for UK and one for US);  Mike Schmidt does not want it 
     distributed by anon ftp.  Martin Guest (UK) has not been contacted about
     distribution.

     David S. suggested to drop ARCO code.  Fiona S. suggested getting a newer 
     version from Chuck Mosher.  David S. also suggested to rethink PICL 
     version of PSTSWM.
     Mark B. indicated that programmers have had a problem with PSTSWM.

     Tony H. suggested future discussions on MPI and HPF versions of these
     applications be made but not for first release.  

     POLMP (by Mark Ashworth at NERC) - in good shape. 
     David W. indicated that Pat Worley can improve PSTSWM for release.
     Should QCD be used?  David S. suggested it be dropped since it is too
     simplistic and not that interesting of an application.  Tony H. suggested
     that it is a "theoretical" benchmark.   Fiona S. suggested that it's 
     to start small and add codes in time. 

                        *** ACTION ITEM ***
     Fiona S. will contact Martin Guest about UK-GAMESS for Parkbench release.
     Some chemistry kernels might be possible also. 

      For June,
         - CFD (BT,LU,SP)
         - Enviro. Modeling (POLMP, PSTSWM?)
         - Oil (C. Mosher's latest ARCO)
         - Chemistry (UK-GAMESS?, MD codes?)

                        *** ACTION ITEM ***
      David W. will oversee the development of first release of Compact Applic.

      At 11:15am, the discussion of the HPF compiler benchmarks began.
      Tony H. asked Fiona S. about IBM's interest in using such benchmarks.

                        *** ACTION ITEM ***
      Jack D. will contact Tom Haupt about his interest in the
      inclusion of his codes in the first release and how to report the results.

      Tony H. suggested that they be left out initially for the
      first release.   A schedule is needed to development first release.

2.   Re-organization of Parkbench WWW pages

      Instructions, Source Codes, Result Database, GBIS Viewer.
      Tony H. proposed an organization of the information.  Jack D. has someone
      funded by CRPC to deal with bug reporting. 

3. Funding of PARKBENCH Activities

      Jack D. has $15,000 from CRPC (Mike Kolatis)
      Berry, Dongarra and Sameh has pre-proposal to NSF.
      Tony H. trying to get vendor-support (Mark Baker funding).
      Tony H. and van der Steen to Europe funding agencies.
      Corporate aid from Sun,  SGI, Intel, IBM, Cray, Convex, HP, Meiko, 
      Parsytec? Funding from NEC, Fujitsu?   

4. General WORK required:

     1) Rules/guidelines for running codes
        SPEC rules by Fiona and Fortran Extensions by David B. in 1 week
     2) Mechanism for bug reporting
     3) Update and stabilize existing codes
        April 7 deadline for contacting application experts for including
        CA's (Worley, Mosher, Guest).  Jack D. and Fiona S. will be attending
        SPEC-HPSC meeting and discuss arrangement for proper release
        of CA's.    Tony H. suggested that ARCO and GAMESS be left out.

                        *** ACTION ITEM ***
         AVL (Fire code) may be included from RAPS.  Tony H. will investigate
         this along with Roger H's LPM2 (PIC code).

     4) Single release
     5) Recommended subset of codes to run

The group had lunch from noon to 1:00pm.
At 1:00pm the following "Timeline" was proposed.
--------------------------------------------------------------------------------
                            TIMELINE for PARKBENCH ACTIVITIES
--------------------------------------------------------------------------------

3/27     lowlevel, kernels, CA's
--------------------------------------------------------------------------------
4/01     UT  : ARCO,kernels,gather software, WWW pages
         ORNL: PSTSWM, WWW pages
         IBM : UK-GAMESS pending HPSC discussion
         USH : lowlevel
         SU   : HPF Benchmarks
--------------------------------------------------------------------------------
4/15     LL+Kernels
         Software ready for US
         Unify makefile
         DOC/Readme
--------------------------------------------------------------------------------
5/1      Actively gathering performance numbers (3 runs - S,M,L)
         Target machines: SP/2, Meiko CS2, Paragon, T3D, Convex, DEC SMP, SGI
         (SP/2 - David B.,  Paragon - UT,  T3D - Ken K., SGI - Horst Simon?,
         Meiko CS2 - Mark B.,  CRI J90 - Charles Grassl)
         Install GBIS at UT, Put #'s into database; Graph BM Info Server
--------------------------------------------------------------------------------
6/1       Announce in places like HPCwire, NA-Digest, other? 
--------------------------------------------------------------------------------

At 1:45pm,  different attendees presented specific results using PARKBENCH
codes.  Tony H. showed performance of Large Integer Sort for a direct-mapped
cache processor.   Looked at bucket sort on NUMA machine with data size <= cache
size and vice-versa.  Jack D. showed a comparison of message passing on 
different platforms: it appeared that T3D using SHMEM was superior. 
David B. then showed new results for the BT NAS Parallel Benchmark code
on a variety of machines. The Cray J90 performance was impressive from a
cost-performance perspective but the DEC 8400 was even better. Results were
presented in gflop/s.

At 2:15pm, Tony H. asked about another meeting before  SC'95.  David B.
suggested the end of August or early September.  A video conference in July 
was a possibility but more than 3 sites is a problem according to Fiona S.  
A teleconference before the release in mid May is a possibility.  Parkbench 
video meeting for May 12 (tentative).   Jack D. will test a "bridge" between
sites for this video meeting.

This Parkbench meeting was adjourned at 2:30 pm by Tony H.