CS 594-30851 Understanding Parallel Architectures:
From Theory To Practice Spring 2001 - 3 credits - Room 211, Claxton

Jack Dongarra, Professor; with help from David Walker, Graham Fagg, Shirley Browne, and Erich Strohmaier.

Email: dongarra@cs.utk.edu
Phone: 865-974-8295
Fax: 865-974-8296
Office hours: Wednesday 11:00 - 1:00, or by appointment

To find out more about this Course click here.
Lecture Notes: (Tentative outline of the class)
  • Jan 10 Lecture on MPI - David Walker
    Slides: Message Passing and MPI
    Slides: Additional material on MPI
    Example: MPI program1
    Example: MPI program2
    Example: MPI program3
    Reading: Overview of Scientific Computing
  • Jan 17 Overview of High Performance Computing
    Slides: Course Introduction
    Slides: Overview of High Performance Computing
    Homework: due: January 24, 2001.
  • Jan 24
    Slides: Introduction to Parallel Architectures and Programming Models
    Reading: Draft notes on Parallel Programming Considerations
  • Jan 31
    Slides: Memory Hierarchy and Cache
    Floating Point Arithmetic
    Slides: Homework: due: February 7, 2001.
  • Feb 7 Performance Eval and OpenMP - Erich Strohmaier
    Slides: Performance Eval and OpenMP
  • Feb 14 Memory Hierarchy and Cache (Continued)
    Slides: Memory Hierarchy and Cache (Continued)
    Homework: due: February 23, 2001.
  • Feb 21 Linear Algebra
    Slides: Linear Algebra
  • Feb 28 Iterative Methods - Victor Eijkhout
    Slides: Sparse Methods
  • Mar 7 Linear Algebra (Continued)
  • Mar 14 Linear Algebra (Continued see Feb 21)
    Slides: Preparing for MetaComputing
  • Mar 21 Spring Break
    Slides:
  • Mar 28 NetSolve and Projects
    Slides: The Grid and NetSolve
    Homework: due: April 11, 2001
  • Apr 4 Parallel Debugging - Shirley Moore
    Slides, Homework, and Notes: due: April 11, 2001
  • Apr 11 Advanced MPI - Graham Fagg
    Slides: pdf file
    Homework: pdf file
  • Apr 18 Metacomputing - Graham Fagg
    Slides: metacomputing 1-2001.pdf metacomputing 1-2001.pdf metacomputing 1-2001.pdf
  • Apr 25 Metacomputing - Graham Fagg
    Slides: Metacomputing 4-2001.pdf
    Help: conn.c conn.h conn.help
  • May 5
    Class Projects Reports
  • May ? Final exam
  • Projects (due May 5, 2001)
    Preliminary list of projects click here.
    Additional Reading Materials

    Message Passing Systems.

    The PVM home page.


    The MPI home page.
    This is the best place to get info on MPI from, including implementations and the MPI forum itself.

    The implementation that you should use, that is installed on the TORC (Tennessee Oak Ridge Cluster) cluster is MPICH .
    A duplex postcript version of the MPI 1.1 API (with thanks to the LAM team) is available.

    Other useful reference material


  • A good introduction to message passing systems.


    J.J. Dongarra, G.E. Fagg, R. Hempl and D. Walker, Chapter in Wiley Encyclopedia of Electrical and Electronics Enginerring,
    to appear. ( postscript version )

    ``Message Passing Interfaces'', Special issue of Parallel Computing , vol 20(4), April 1994.

  • A paper by members of the PVM team on the differences between PVM and MPI.

    Geist, G.A, J.A. Kohl, P.M. Papadopoulos, `` PVM and MPI: A Comparison of Features '', Calculateurs Paralleles , 8(2), pp. 137--150, June, 1996.

  • Papers by members of the MPI team on the differences between PVM and MPI.

    ``Why are PVM and MPI So Different'', William Gropp and Ewing Lusk (submitted to The Fourth European PVM - MPI Users' Group Meeting)

    and

    ``PVM and MPI are completely different'', William Gropp and Ewing Lusk, to appear in the journal Future Generation Computer Systems, 1998.


    Standard Books on Message Passing

    ``MPI - The Complete Reference, Volume 1, The MPI-1 Core, Second Edition'',
    by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, Jack Dongarra, MIT Press, September 1998, ISDN 0-262-69215-5.

    ``Using MPI,''
    by William Gropp, Ewing Lusk, and Anthony Skjellum, published by MIT Press, October 1994; ISBN 0-262-57104-8.

    ``MPI: The Complete Reference - 2nd Edition: Volume 2 - The MPI-2 Extensions'',
    by William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, William Saphir, and Marc Snir, published by The MIT Press, September, 1998; ISBN 0-262-57123-4.

    Class Roster If your name is not on the list or some information is incorrect, please send mail to TA: .
    Student Names and Their Email addresses





    On-line Documentation and Information about Machines

  • Cray
  • IBM RS6000
  • Intel
  • Intel ASCI Red Paragon
  • SGI Power Challenge
  • Solaris Threads page
  • Catalog of Commercial Hardware and Software Vendors
  • Convex
    • Exemplar
    • Michielse, P. Programming the Convex Exemplar Series SPP system. Parallel Scientific Computing. First International Workshop, PARA '94. Proceedings. Lyngby, Denmark, 20-23 June 1994). Edited by: Dongarra, J.; Wasniewski, J. Berlin, Germany: Springer-Verlag, 1994. p. 374-82.
  • Cray Research
  • Digitial Equipment Corporation ( System Info
  • Hewlett-Packard
  • IBM
  • Sequent
    • Symmetry 5000
    • NUMA-Q
    • Raetz, G.M., Sequentz, G.M., Sequent general purpose parallel processing system, Northcon/87. Conference Record. Portland, OR, USA, 22-24 Sept. 1987)
  • Silicon Graphics
    • Power Challenge
    • Power Challenge (Techical Report)
    • Challenge XL
    • Galles, M.; Williams, E. Performance optimizations, implementation, and verification of the SGI Challenge multiprocessor. Proceedings of the Twenty-Seventh Hawaii Internation Conference on System Sciences Vol. I: Architecture, Wailea, HI, USA, 4-7 Jan. 1994, Edited by: Mudge, T.N.; Shriver, B.D. Los Alamitos, CA, USA: IEEE Comput. Soc. Press, 1994. p. 134-43.
    • Power Series
  • Sun Microsystems
    • Other Parallel Information Sites
  • NHSE - National HPCC Software Exchange
  • Netlib Repository at UTK/ORNL
  • BLAS Quick Reference Card
  • LAPACK
  • ScaLAPACK
  • GAMS - Guide to Available Math Software
  • Center for Research on Parallel Computation (CRPC)
  • Supercomputing & Parallel Computing: Conferences
  • Supercomputing & Parallel Computing: Journals
  • High Performance Fortran (HPF) reports
  • High Performance Fortran Resource List
  • Fortran 90 Resource List
  • CMU's list of supercomputing and parallel computing resources
  • Major Science Research Institutions from Caltech
  • Message Passing Interface (MPI) Forum
  • High Performance Fortran Forum
  • OpenMP
  • PVM
  • Parallel Tools Consortium
  • DoD High Performance Computing Modernization Program
  • DoE Accelerated Strategic Computing Initiative (ASCI)
  • National Computational Science Alliance
    • Related On-line Textbooks
  • Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM Publication, Philadelphia, 1994.
  • PVM - A Users' Guide and Tutorial for Networked Parallel Computing, MIT Press, Boston, 1994.
  • MPI : A Message-Passing Interface Standard
  • LAPACK Users' Guide (Second Edition), SIAM Publications, Philadelphia, 1995.
  • MPI: The Complete Reference, MIT Press, Boston, 1996.
  • Using MPI: Portable Parallel Programming with the Message-Passing Interface by W. Gropp, E. Lusk, and A. Skjellum
  • Parallel Computing Works, by G. Fox, R. Williams, and P. Messina (Morgan Kaufmann Publishers)
  • Computational Science Education Project TextBook.
  • Designing and Building Parallel Programs. A dead-tree version of this book is available by Addison-Wesley.
  • High Performance Fortran (HPF), a course offered by Manchester and North High Performance Computing Training & Education Centre, United Kingdom
  • Parallel Processing Laboratory, Colorado School of Mines

    • For performance analysis:

  • Raj Jain, The Art of Computer Systems Performance Analysis. John Wiley, 1991.

    • Papers on performance analysis tools:

  • Ruth A. Aydt, "The Pablo Self-Defining Data Format," November 1997, click here.
  • Daniel A. Reed, Ruth A. Aydt, Tara M. Madhyastha, Roger J. Noe, Keith A. Shields, and Bradley W. Schwartz, "Pablo: An Extensible Performance Analysis Environment for Parallel Systems", November 1992, click here.
  • Jeffrey K. Hollingsworth, Barton P. Miller, Marcelo J. R. Gongalves, Oscar Naim, Zhichen Xu and Ling Zheng, "MDL: A Language and Compiler for Dynamic Program Instrumentation", International Conference on Parallel Architectures and Compilation Techniques, San Francisco, CA, November 1997, click here.
  • Barton P. Miller, Mark D. Callaghan, Jonathan M. Cargille, Jeffrey K. Hollingsworth, R. Bruce Irvin, Karen L. Karavanic, Krishna Kunchithapadam and Tia Newhall. "The Paradyn Parallel Performance Measurement Tools", IEEE Computer 28(11), (November 1995). click here.
  • Steven T. Hackstadt and Allen D. Malony, "Distributed Array Query and Visualization for High Performance Fortran, February 1996, click here.
  • Jerry Yan and Sekhar Sarukkai and Pankaj Mehra, "Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS toolkit", Software Practice and Experience 25(4), April 1995, 429--461

    • Jack Dongarra
    1/8/2000