Subject: Marc Snir's paper

From: [email-masked] (William Gropp)
Date: Wed, 9 Dec 92 15:32:39 CST

Thanks to Marc for laying out many of the issues so clearly.  We find
that we agree with many and disagree with some.

Goals:
(1) We think that those most in need of a standard are those writing libraries,
and therefor the standard should cater to their needs as well as those of
end users.  It will be easy to design simple application programming
interfaces on top of the MPI standard.

(2) We strongly agree with this point (allow efficient communication).

(3) We are not sure what "allow (but no mandate)" for heterogeneous
communications means.  If this means that vendors of homogeneous massively
parallel systems may implement only the homogeneous operations, then we 
agree, as long as the heterogeneous operations are defined by the MPI
standard.

We agree in principle with the others.

Framework:
We also find the described framework useful for thinking about the operations,
but not necessarily the right structure for the MPI.

Issues:
An exchange operation can be useful.  At least two MPP vendors (TMC and Intel)
provide such operations.
We also agree an leaving out RPC, active messages, etc, as these are topics
of rapidly moving research.

We believe that there may be something more inclusive than channels that can
serve as the basis for specifying high-performance communication operations.

We do not find the discussion of "local versus global" termination to be
a good way to consider the issues of blocking, nonblocking, and synchronous
communication.  In particular, blocking and nonblocking refer to the
state of the buffer containing the message on return from the routine
(be it send, receive, or wait), and synchronous (in the MPI proposal) really
refers to an alternate protocol for the delivery of a message.

We believe that relying on an exception handling mechanism for error handling
will make the system too fragile.  We propose allowing the return of error
conditions.  An alternate proposal allows the user to specify an error routine
to be called in the event of an error.  Note that library implementors may
need to handle errors explicitly.

Correctness Criteria:
We believe that it is important to allow the user to determine (at least a 
bound on) the amount of resources available.  Further, it MAY be appropriate
to specify a minimum size that is supported (otherwise zero may be consistent
with the letter of the standard).  

Communication Parameters:
Our draft implementation includes an extension to the current proposal that
permits the communication of messages of mixed datatype (we interpret "type"
on page 7 to mean "datatype" rather than "message type").  This is a simple
extension to the data description vector that allows for heterogeneous
communication.

Groups and context:
We agree that the collective communication committee should be the one
to discuss these issues.  We do feel that the destination and source
processor ids should not be relative to some "current" group.  Note that
such a functionality (with some restrictions) can be provided by a simple
layer on top of MPI.

For an example of a problem with making processor id's relative to a
current "group", consider recv(source=DONT_CARE,...).  Does this mean
only in the current group?  What is the result of MPI_INFOS() on the received
message?  What if two processors A and B are both in two groups alpha and
beta, but with different rankings (ie., A = 2 and B = 5 in alpha, and
A = 12 and B = 0 in beta).  Now A (current group alpha) sends to B (current
group beta).  B does recv(source=DONT_CARE,...).  Does B receive this message?
What is the value of MPI_INFOS()?  If B doesn't receive this message, then
how is a communication context different from a group that is a copy of the
current group (which would seem to supply the same semantics)?

Matching of send and receive:
We believe that there should be two types of matching operations: a simple
(efficient) one on message type, and a general one using a user-defined
predicate (matching function).  Our experience has been that once the users
have "select by source", they'll then want "select by source range and/or type
range and/or message size and/or ...".

We also agree that truncation of a received message into a too-short buffer
should be an error (or at least detectable!).

Syntax:
We agree with multiple function names rather than multiple modes.  It is
easier to provide a efficient implementation and clear semantics in this
case (for example, having the return code always have the same meaning
independent of the input arguments).  Note also that it is easy, given
a multiple function names approach, to produce one-routine-does-everything
interfaces but not vice-versa.

Miscelania:
We must have a non-blocking AWAIT.
We'd like to AWAIT several messages.

We must have a PROBE.

We agree that the issue of returning data on received messages (particularly
those done with nonblocking operations) is very important.  One approach
(used by one MPP vendor) is to ask for this info by handle, and then
explicitly free the handle.  This eliminates unnecessary global state 
(the currently most recently received message) from the semantics, and 
makes it possible to easily describe what exactly is returned by these
"INFO" routines.

Bill Gropp and Rusty Lusk