Subject: Re: Precise meaning of MPI_COMMIT

From: [email-masked]
Date: Thu, 22 Apr 93 17:32:09 PDT

David Walker writes:

> ... Once the
> description of the message (using MPI_ADD_XXXX) and the composition of
> the message (using MPI_COMMIT_BUFFER, or better MPI_COMPOSE) are
> clearly separated then it is no longer clear why you need a
> commit/compose routine, since the assembly/disassembly of the message
> would be done by the send/receive routines.
> 
> So I propose that the draft makes clear the difference between
> describing a message and composing it, and that MPI_COMMIT_BUFFER be
> deleted.

Commit is included to permit optimization.

In the absence of an explicit commit, the buffer descriptor must
always be maintained in some format that allows further MPI_ADD_XXXX
calls to be made.  With an explicit commit, MPI is permitted to
translate the buffer descriptor to a format that can be used quickly
by send/receive, but may not be modifiable.  I am not aware of current
message passing systems where this is done, but there is enough
precedent in other areas to convince me that commit is a good thing to
include.

I oppose deleting the commit.

------

David continues:

> > (Rik wrote..)
> >This solves David's problem on the send side.
> >However, it does NOT solve the corresponding problem on the receive
> >side of David's application.  
> >The receive buffer descriptor must be generated before the receive
> >call, thus before any of the data is transferred to user variables,
> >thus the descriptor cannot depend on contents of the message.
> 
> Isn't this just the usual problem that you have when you don't know
> how many things you're going to receive. You just have to provide
> a buffer that's big enough to handle the largest number of things that
> you expect to receive.

No code was shown for the receive side.  I assumed that it would do
a scatter similar to the gather on the send side.  Perhaps that was
not the case?  Still reading between the lines, I now get the feeling
that the receive side wants to get two integers followed by an unknown
number of contiguous doubles, in which case I agree that just
providing a big buffer is adequate.

> My example was trying to use buffer descriptors to send a message
> containing mixed data types. I want to send everything in one message
> in the (possibly mistaken) belief that it will be faster to send
> everything in one message rather than sending one message for the
> integer counts and another one for the particle positions. If the time
> to send a message is dominated by the time to handle the buffer
> descriptors then nobody will use them. 
>
> Do you think my code would run
> faster if I cast the counts as doubles, pack them and the particle
> positions into a contiguous buffer myself and then communciate them
> using MPI_SENDC/MPI_RECVC (the simple version for contiguous buffer)?

Yes.

In gross generality, I would expect buffer descriptors to be fast
if and only if they contain substantially fewer components than the
number of data elements to be transferred.  Thus it would be better
to describe a couple of integers and a couple of indexed vectors
(a total of 4 descriptor components) than to describe each
data element separately.  

Is it faster to pack/unpack the data yourself?  Depends.  If the
descriptor gets reused lots of times, then it is conceivable that
say describing an indexed vector (and thus allowing send to gather the
data) would be faster than gathering the data yourself and sending it
as a contiguous buffer.  However, if the descriptor gets used for only
a single send, then for sure it's gonna be slower to generate the
descriptor and make send interpret it, than to just copy the data
yourself.  I am not clear on whether David's application intended to
reuse the descriptor.

> Well can we have some examples from real applications in which the
> generality and flexibility offered by MPI buffer descriptors can be
> used, and still have the code run reasonably efficiently.

For indexed vectors, any application with a repeating communication
pattern.  Extreme example, Dmitri Mavriplis's FLO72 code (NASA/ICASE),
hundreds of identical iterations.  Not so extreme, molecular dynamics
with neighbor recalculation say every 20 time steps (and thus
repeating the comm pattern 20 times for each determination).  Say,
DISCOVER, AMBER, CHARMM.  For strided vectors, any application wanting
to send an array slice containing at least a few tens of data elements.
Wild guesses, of course...

--Rik
----------------------------------------------------------------------
[email-masked] (alias 'd39135')   Rik Littlefield
Tel: 509-375-3927                         Pacific Northwest Lab, MS K1-87
Fax: 509-375-6631                         P.O.Box 999, Richland, WA  99352

---

From: James Cownie <[email-masked]>
Date: Fri, 23 Apr 1993 04:44:02 +0100

My understanding (for what that's worth) of the purpose of the commit
is that it allows the processing of the buffer descriptor itself. It
does NOT bind any of the actual data, that is always done at the point
of sending or receiveing the message.

I could imagine it being very useful to have the commit if one had
hardware similar to an IBM channel. The commit point would be the time
when you could compile the channel program, as at that point you know
all of the properties of the dat accesses required to compose the
message. This would bind the addresses of the data, but the actual
data movement would occur when the channel program were executed (i.e.
on the actual send/recv).

So the commit is telling the system that it now knows all about the
layout of the message. It says nothing about the data content, which
one assumes will change each of the many times the committed message
descriptor is used.

-- Jim
James Cownie 
Meiko Limited			Meiko Inc.
650 Aztec West			Reservoir Place
Bristol BS12 4SD		1601 Trapelo Road
England				Waltham
				MA 02154

Phone : +44 454 616171		+1 617 890 7676
FAX   : +44 454 618188		+1 617 890 5042
E-Mail: [email-masked]   or    [email-masked]