CS 594-30851 Understanding Parallel Architectures:
From Theory To Practice Spring 2003 - 3 credits - Room 211, Claxton

Jack Dongarra; with help from Shirley Moore, Victor Eijkhout, Graham Fagg, and Dan Terpstra

Email: dongarra@cs.utk.edu

Phone: 865-974-8295

Fax: 865-974-8296

Office hours: Wednesday 11:00 - 1:00, or by appointment

TA: Kenneth Roche, roche@cs.utk.edu

O : 351 Claxton Complex, 974-9989

OH: 10am-11am TR, or by request

 

Class Roster

 If your name is not on the list or some information is incorrect, please send mail to TA: .
cremus@cs.utk.edu , Cathy Remus

rzw@ornl.gov , Jane Razumovskaya

efuentes@cs.utk.edu , Erika Fuentes

cheng@cs.utk.edu , Chang Cheng

burton@cs.utk.edu , Jeff Burton

larkin@cs.utk.edu , Jeff Larkin

min@cs.utk.edu , Min Zhou

wgardner@cs.utk.edu , Bill Gardner

 

To find out more about this Course click here.

cover imageBook for Class:

 

 

The Sourcebook of Parallel Computing, Edited by Jack Dongarra, Ian Foster, Geoffrey Fox, William Gropp, Ken Kennedy, Linda Torczon, Andy White, October 2002, 760 pages, ISBN 1-55860-871-0, Morgan Kaufmann Publishers.

 

 

 

 

 

 

Lecture Notes: (Tentative outline of the class)

 

  1. January 15
    1. Introduction (slide set 1, slide set 2)
    2. Chapter 1 pp 3-13
    3. Homework: due January 29th, 2003
  2. January 22
    1. MPI (Graham Fagg) (slide set 3, slide set 4, slide set 5)
    2. Chapter 10 pp 313 – 330
    3. Chapter 11 pp 331 – 357
    4. Homework: due noon February 5th, 2003 email to Graham Fagg (fagg@cs.utk.edu)
    5. Example for pi and code
    6. additional information here
  3. January 29
    1. Programming, Architectures (slide set)
    2. Chapter 2 pp 15-41
    3. Chapter 3 pp 45 – 70
  4. February 5
    1. High Performance Computers (slide set)
    2. Chapter 3 pp 43 – 70
    3. Homework: due February 19th, 2003
  5. February 12
    1. Memory Hierarchy (slide set)
    2. Chapter 3 pp 43 – 70
    3. Homework: due February 26th, 2003
  6. February 19
    1. Linear Algebra (slide set)
    2. Chapter 20
    3. Homework: due March 5th, 2003
  7. February 26
    1. Grid Computing (Graham) (slide1, slide2, slide3, slide4)
    2. Chapter 14 pp 409 – 442
    3. Homework: due March 12th, 2003
  8. March 5
    1. Iterative Methods (Victor) (slide set)
    2. Chapter 20
    3. Homework: due March 26th, 2003
    4. Notes
  9. March 12
    1. Linear Algebra (slide set)
    2. Chapter 20

      March 19 – Spring Break

  1. March 26
    1. Grid Computing (Graham) (slide1, slide2)
    2. Chapter 14 pp 409 – 442
    3. Homework: due April 9th, 2003
  2. April 2
    1. Linear Algebra (slides)
    2. Chapter 20
  3. April 9
    1. Tools (Shirley Moore) (slide1, slide2)
    2. Chapter 15 pp 443 – 467
    3. Homework: due April 23rd, 2003
  4. April 16
    1. Linear Algebra (slides)
    2. Chapter 20
  5. April 23
    1. PAPI (Dan Terpstra) (slides)
  6. April 30 – Last Class
    1. Reports
    2. We will start at 12:00 noon and plan to finish by 3:00

·  Projects (due April 2003)
Preliminary list of projects click here.

Additional Reading Materials

Message Passing Systems.

The PVM home page.


The MPI home page.
This is the best place to get info on MPI from, including implementations and the MPI forum itself.

The implementation that you should use, that is installed on the TORC (Tennessee Oak Ridge Cluster) cluster is MPICH .
A duplex postscript version of the MPI 1.1 API (with thanks to the LAM team) is available.

Other useful reference material

 

·  A good introduction to message passing systems.


J.J. Dongarra, G.E. Fagg, R. Hempl and D. Walker, Chapter in Wiley Encyclopedia of Electrical and Electronics Engineering,
to appear. ( postscript version )

``Message Passing Interfaces'', Special issue of Parallel Computing , vol 20(4), April 1994.

·  A paper by members of the PVM team on the differences between PVM and MPI.

Geist, G.A, J.A. Kohl, P.M. Papadopoulos, `` PVM and MPI: A Comparison of Features '', Calculateurs Paralleles , 8(2), pp. 137--150, June, 1996.

·  Papers by members of the MPI team on the differences between PVM and MPI.

``Why are PVM and MPI So Different'', William Gropp and Ewing Lusk (submitted to The Fourth European PVM - MPI Users' Group Meeting)

and

``PVM and MPI are completely different'', William Gropp and Ewing Lusk, to appear in the journal Future Generation Computer Systems, 1998.

 

Standard Books on Message Passing

``MPI - The Complete Reference, Volume 1, The MPI-1 Core, Second Edition'',
by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, Jack Dongarra, MIT Press, September 1998, ISDN 0-262-69215-5.

``Using MPI,''
by William Gropp, Ewing Lusk, and Anthony Skjellum, published by MIT Press, October 1994; ISBN 0-262-57104-8.

``MPI: The Complete Reference - 2nd Edition: Volume 2 - The MPI-2 Extensions'',
by William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, William Saphir, and Marc Snir, published by The MIT Press, September, 1998; ISBN 0-262-57123-4.

On-line Documentation and Information about Machines

·  Cray

·  IBM RS6000

·  Intel

·  Intel ASCI Red Paragon

·  SGI Power Challenge

·  Solaris Threads page

·  Catalog of Commercial Hardware and Software Vendors

·  Convex

      • Exemplar
      • Michielse, P. Programming the Convex Exemplar Series SPP system. Parallel Scientific Computing. First International Workshop, PARA '94. Proceedings. Lyngby, Denmark, 20-23 June 1994). Edited by: Dongarra, J.; Wasniewski, J. Berlin, Germany: Springer-Verlag, 1994. p. 374-82.

·  Cray Research

·  Digitial Equipment Corporation ( System Info

·  Hewlett-Packard

      • Thomas B. Alexander, Kenneth G. Robertson, Deal T. Lindsay, Donald L. Rogers, John R. Obermeyer, John R. Keller, Keith Y. Oka, and Marlin M. Jones II. Corporate Business Servers: An Alternative to Mainframes for Business Computing. Hewlett-Packard Jornal, June 1994, pages 8-33.
      • HP-9000 K400, HP-9000 K200
      • HP-9000 T500

·  IBM

·  Silicon Graphics

      • Power Challenge
      • Power Challenge ( Techical Report)
      • Challenge XL
      • Galles, M.; Williams, E. Performance optimizations, implementation, and verification of the SGI Challenge multiprocessor. Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences Vol. I: Architecture, Wailea, HI, USA, 4-7 Jan. 1994, Edited by: Mudge, T.N.; Shriver, B.D. Los Alamitos, CA, USA: IEEE Comput. Soc. Press, 1994. p. 134-43.
      • Power Series

·  Sun Microsystems

Other Parallel Information Sites

·  NHSE - National HPCC Software Exchange

·  Netlib Repository at UTK/ORNL

·  BLAS Quick Reference Card

·  LAPACK

·  ScaLAPACK

·  GAMS - Guide to Available Math Software

·  Center for Research on Parallel Computation (CRPC)

·  Supercomputing & Parallel Computing: Conferences

·  Supercomputing & Parallel Computing: Journals

·  High Performance Fortran (HPF) reports

·  High Performance Fortran Resource List

·  Fortran 90 Resource List

· 

·  Major Science Research Institutions from Caltech

· 

·  Message Passing Interface (MPI) Forum

·  High Performance Fortran Forum

·  OpenMP

·  PVM

·  Parallel Tools Consortium

·  DoD High Performance Computing Modernization Program

·  DoE Accelerated Strategic Computing Initiative (ASCI)

·  National Computational Science Alliance

Related On-line Textbooks

·  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM Publication, Philadelphia, 1994.

·  PVM - A Users' Guide and Tutorial for Networked Parallel Computing, MIT Press, Boston, 1994.

·  MPI : A Message-Passing Interface Standard

·  LAPACK Users' Guide (Second Edition), SIAM Publications, Philadelphia, 1995.

·  MPI: The Complete Reference, MIT Press, Boston, 1996.

·  [new]Using MPI: Portable Parallel Programming with the Message-Passing Interface by W. Gropp, E. Lusk, and A. Skjellum

·  Parallel Computing Works, by G. Fox, R. Williams, and P. Messina (Morgan Kaufmann Publishers)

·  Computational Science Education Project TextBook.

·  Designing and Building Parallel Programs. A dead-tree version of this book is available by Addison-Wesley.

·  High Performance Fortran (HPF), a course offered by Manchester and North High Performance Computing Training & Education Centre, United Kingdom

For performance analysis:

·  Raj Jain, The Art of Computer Systems Performance Analysis. John Wiley, 1991.

Papers on performance analysis tools:

·  Ruth A. Aydt, "The Pablo Self-Defining Data Format," November 1997, click here.

·  Daniel A. Reed, Ruth A. Aydt, Tara M. Madhyastha, Roger J. Noe, Keith A. Shields, and Bradley W. Schwartz, "Pablo: An Extensible Performance Analysis Environment for Parallel Systems", November 1992, click here.

·  Jeffrey K. Hollingsworth, Barton P. Miller, Marcelo J. R. Gongalves, Oscar Naim, Zhichen Xu and Ling Zheng, "MDL: A Language and Compiler for Dynamic Program Instrumentation", International Conference on Parallel Architectures and Compilation Techniques, San Francisco, CA, November 1997, click here.

·  Barton P. Miller, Mark D. Callaghan, Jonathan M. Cargille, Jeffrey K. Hollingsworth, R. Bruce Irvin, Karen L. Karavanic, Krishna Kunchithapadam and Tia Newhall. "The Paradyn Parallel Performance Measurement Tools", IEEE Computer 28(11), (November 1995). click here.

·  Steven T. Hackstadt and Allen D. Malony, "Distributed Array Query and Visualization for High Performance Fortran, February 1996.

·  Jerry Yan and Sekhar Sarukkai and Pankaj Mehra, "Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS toolkit", Software Practice and Experience 25(4), April 1995, 429--461

Jack Dongarra
1/7/2003