Subject: Re: mpi-pt2pt: unsafe program? (short)

From: [email-masked] (Leslie Hart)
Date: Mon, 12 Apr 93 12:54:38 MDT

> Subject: Re: mpi-pt2pt: unsafe program? (short) 
> Date: Fri, 09 Apr 93 09:44:22 MST
> From: [email-masked]
> 
> Lyndon asks if the program
> 
> Process 1                 Process 2
> 
> send(2, tag=1)            recv(1, tag=2)
> send(2, tag=2)            recv(1, tag=1)
> 
> is unsafe. The answer is yes. With no buffering, process 2 cannot
> complete the first recv, since process 1 has not sent the first msg yet
> and process 1 cannot complete the first send, since process 2 has
> not posted a receive for it. Even if buffering is available, the program
> can still hang with large enough messages. The only safe way to
> write this program is for one of the processes to post its first
> operation in unblocked mode.

I have a similar question to Lyndon's.  Given that I don't think MPI has
yet specified a lower limit on the number of outstanding requests that may
exist, is the unblocked version safe.  Suppose you can only have one
outstanding request.  Again, clearly only one outstanding request is too
few, but how many is enough?  I would personally expect "most" programs
written with non-blocking receives to work, but they may well be unsafe.
In an algorithm where many boundary exchanges are happening there may be
a lot of pending non-blocking receives and this is the problem I think of
when the question of safe versus unsafe arises.

Regards,
Leslie Hart ([email-masked])

---

From: [email-masked] (Peter Rigsbee)
Date: Mon, 12 Apr 93 15:25:03 CDT

> when the question of safe versus unsafe arises.

I find these discussions quite frustrating and mostly theoretical.  I'd 
suggest we note these as issues and move on.  Although opposed to recent 
political trends, I'd suggest we take the approach of letting the 
marketplace settle issues like this.

The Fortran 77 standard does not specify a minimum number of COMMON blocks
that a compiler must support.  There is probably an implicit minimum of two 
(blank common and a named common block).  So anyone could write and market a 
standard-conforming Fortran compiler that only lets you have two common 
blocks per compilation unit.  

How many users worry about this?   How many users jam all their data into 
one named COMMON block so it will be compilable by such a brain-dead 
compiler?  Not many.  Why?  Because such compilers aren't going to survive 
in the marketplace -- they will either be fixed (by increasing the limit), 
or they won't have any users.  So someone trying to write portable Fortran 
uses a "reasonable" number of COMMON blocks (i.e., a number that is 
supported by all the systems of immediate interest) and figures he'll wait 
in case he hits a compiler limit with some new machine before worrying about 
it further.

The same thing will be true of MPI.  Who cares if someone writes an MPI
implementation that supports at most one active send and receive in the
entire system?  If there is an official MPI test suite, the test suite will
probably fail.  Even if there isn't, there won't be any MPI applications 
that will run.  Either that implementation will be fixed, or it will die on 
its own.  The MPI standard doesn't have to solve this problem, the marketplace
will.

Taking a less extreme example, assume a new version of MPI is developed that
can't run a code that runs on every other MPI system.  I suggest that the
developer of that code will point this out to the new MPI vendor.  Ideally, 
the end result of this discussion is that the final application runs 
everywhere.  Maybe this means the new MPI version is changed, or the 
application is changed, or both are changed.  There may also be cases where 
the result is that the application is not ported to the new version of MPI, 
perhaps to the detriment of both vendor and developer, but maybe not.  
(For example, the new system may simply be too small to run that particular 
application.  In this case, the presence of the smaller limit helped 
identify this situation.)

The major problem with setting bounds for these things is one of coming to 
agreement.  Using Leslie's question as an example, I suggest that we'd all 
agree that only one outstanding request is too few.  And we'd probably agree 
that having 1,000,000 is more than enough.  But trying to agree on a single
minimum value (or even minimum order of magnitude) between 1 and 1,000,000 
will take forever.  And whatever we agreed on would be wrong for some 
situations, discouraging MPI's implementation.

The other problem is that of units.  I think this came up in the discussion 
of enquiry functions.  IMO, we don't want MPI forcing a particular 
implementation.  Let us consider the question of how do you define the 
maximum number of outstanding requests.  One MPP vendor might want to define 
a value which means "number per CPU".  Another would want to define his 
as "number per thread".  A third uses a shared memory pool and wants to 
have a value which means "number for system as a whole".  A vendor with a 
network implementation might want to support a different value for each 
machine in the network.  Which unit(s) do you use for the definition?  
And how will this effect the other (and future) implementations?

In summary, I think questions about the minimum maximum number of outstanding 
requests, or how many contexts and/or groups should be supported, or about 
whether or not a particular code sequence is safe for all sizes of data and
all possible implementation styles are interesting questions that vendors 
and users should think about.  But if MPI tries to choose and dictate the 
answers, it will take forever to finish and be less widely accepted and 
implemented.

	- Peter Rigsbee