Subject: Precise meaning of MPI_COMMIT

From: [email-masked] (David Walker)
Date: Wed, 21 Apr 1993 15:27:38 -0400

Suppose I have an array of particles:

struct { int type;
         double position[3];
         double velocity[3];
} particle[1000];

and I want to take the positions of all the particles of type=1 and type=2 and 
send them somewhere else. In addition I want to also send the number of 
particles of each type that I'm sending. And I only want to send one message. 
Can MPI do this?

Here's one approach that I think doesn't work according to the current
version of MPI.

      int count1=0;
      int count2=0;
      mpi_create_buffer (handle, MPI_PERSISTENT);
      mpi_add_contiguous(handle, count1, 1, MPI_INT);
      mpi_add_contiguous(handle, count2, 1, MPI_INT);
      for (i=0;i<1000;i++) {
	 if ( particle[i].itype == 1 ) {
	    count1++;
	    mpi_add_contiguous (handle, particle[i].position, 3, MPI_DOUBLE);
         } else if ( particle[i].itype == 2 ) {
	    count2++;
	    mpi_add_contiguous (handle, particle[i].position, 3, MPI_DOUBLE);
         }
      }
      mpi_commit_buffer (handle);

This doesn't work because MPI only guarantees that the message associated
with the handle is no longer volatile once  mpi_commit_buffer returns. So
the above example will fail if an implementation actually copies count1 and
count2 to the message buffer when mpi_add_contiguous is called. The example 
will work in an implementation that copies the data into the message buffer
only when mpi_commit_buffer is called.

You might ask why not put 

      mpi_add_contiguous(handle, count1, 1, MPI_INT);
      mpi_add_contiguous(handle, count2, 1, MPI_INT);

immediately before the call to mpi_commit_buffer. Because in this case its 
impossible to construct the buffer for the receiving process unless it already
knows what count1 and count2 are.

So my suggestion is that maybe we should stipulate that no data gets moved
around until the call to mpi_commit_buffer.

BTW this is not a contrived example. You need to do something similar in 
molecular dynamics simulations in which you cells containing differing
numbers of particles between processors.

Regards,
David

--------------------------------------------------------------------------
| David W. Walker                 |   Office   : (615) 574-7401          |
| Oak Ridge National Laboratory   |   Fax      : (615) 574-0680          |
| Building 6012/MS-6367           |   Messages : (615) 574-1936          |
| P. O. Box 2008                  |   Email    : [email-masked] |
| Oak Ridge, TN 37831-6367        |                                      |
--------------------------------------------------------------------------

---

From: Adam Greenberg <[email-masked]>
Date: Wed, 21 Apr 93 15:53:56 EDT

Date: Wed, 21 Apr 1993 15:27:38 -0400
   From: [email-masked] (David Walker)

   struct { int type;
	    double position[3];
	    double velocity[3];
   } particle[1000];

	 int count1=0;
	 int count2=0;

I'm a bit perturbed by this.  Two issues:

1.  I agree with your suggestion.  I don't beleive we shoudl be in the business
of producing a specification that leaves holes so large that the hapless user
need worry about issues like this.

2.  I admit to being puzzled about this usage.  I would think that you could
code your solution differently so that this contorted logic would not be
required.

moose