Subject: Marc Snir's paper

From: [email-masked]
Date: Thu, 3 Dec 1992 16:19:30 +0100

I would like to make some comments on the document Marc Snir has
distributed.

1. I agree on all points in the list of goals. I find it especially 
   important to define something which is not too far away from common
   practice, otherwise nobody would follow us. A point which is missing
   on the list (perhaps because it is too obvious): the interface should
   not contain more functions than are really necessary. We must
   avoid to end up with a mixed bag which contains multiple mechanisms
   for the same thing.

2. I like Marc's typification scheme for message passing functions based
   on the 4 suboperations. I understand those suboperations more as a
   way to describe the semantics rather than the implementation 
   of the (compound) functions which the user will see.
   
   I'm not sure that the channel based operations are necessary on the
   user level. Basically there are the two addressing schemes for
   message passing: either a message is sent to a named process, or
   it is sent along a named connection (channel), in which case the
   processes don't even need to have identifiers. We had a long
   discussion here in Europe on which scheme is better. In the end it
   turns out that both are more or less equivalent. Most existing
   user interfaces tend towards the named process approach.

   The only reason for supporting both concepts in the same interface
   seems to be the possibility of some additional optimization. If
   suboperations (1) and (4) are separated, ressource allocation and
   deallocation can be saved. However, I think that a clever operating
   system can detect frequently used communication paths
   automatically, and it can establish a permanent link for them.
   (Link caching)
   
3. We will have various message passing mechanisms, especially
   synchronous and asynchronous communication (global/local send
   termination). I strongly recommend that the corresponding 
   send and receive functions must match. On one hand this gives the
   opportunity for some implementation optimization, and also it
   might be used for correctness checking.

4. As Marc said, in MPI1 termination is either local or global, at
   the user's choice. I really think that this is necessary, and a 
   restriction on either synchronous or asynchronous communication
   would be hard to accept.

5. Marc suggests that there should be no "unsuccessful return" of
   a message passing operation. I agree.

6. On page 7, Marc discusses the different protocols for message
   passing. I'm not sure that we want to define this in detail, since
   it is very much implementation dependent. On the other hand we
   will have to accept the fact that a program will run on a machine
   with huge system buffers and not on a smaller system. Somehow
   this is true with other system resources, also.
   
7. Page 7, further down: if we deal with heterogeneous networks,
   a contiguous message buffer may contain sections with different
   data type. The reason is always that one wants to send as few
   messages as possible (startup minimization). Personally I prefer
   to have a separate library call before the send which specifies
   the contents of the message buffer. This information is then
   used or ignored by the send routine depending on the data
   representations in the sender and receiver processors.

8. If our goal is to define a scalable interface, setting up groups
   by specifying member lists in all participating processes is
   a problem. This is especially so for the handling of very
   regular group structures, like grids or tori, since these
   will most likely be used by applications with very large process
   numbers. I think the Virtual Topology subcommittee has to deal with
   this problem.

9. Communication context: I very much favor option 1. (i.e. context
   switch by some special functions) against explicitely specifying
   the context in a (collective) communication statement. I do not
   see that the explicit specification would lead to less user errors,
   and it would add another item to the argument lists.

10. Receive criteria: I do not see a good point for having "don't care"
    addresses in sends. For "don't care" in receive I prefer named
    constants over "magic values". Be sure not to use arguments
    for both input to and output from a function, especially if it
    may contain wildcards on input. There is no better way to make
    sure your constants get overwritten!

11. Matching of data: In Fortran I think option 2. is acceptable.
    If the receive buffer is too short, this is a user error.
    In C there is another option which should be supported by the
    interface: if the receive buffer pointer is NULL, the system
    allocates one with the right size automatically.

12. Syntax: multiple modes like in MPI1 are okay with me, but I
    hate to see character strings there. Talk to any implementor who
    cares about efficiency, and you know what I mean.

13. Also, in Fortran I find it better to use procedure calls for most
    functions.

14. Miscelania: 
    a) We will need a nonblocking AWAIT operation. This is a great
       help in organizing the overlap of computation and communication.
    b) How could we survive without a PROBE function? 
    c) What about providing an inquiry subroutine, which returns
       the message length, the sender, and the type of the
       "last message" which was received? For non-blocking messages
       this is not well defined, but a solution should be possible.

This has become a longer message than I had planned. Sorry for bothering
you all so much!

Rolf Hempel