Collective communications transmit data among all processes in a group specified by an intracommunicator object. One function, the barrier, serves to synchronize processes without passing data. MPI provides the following collective communication functions. collective communication functions.
o Barrier synchronization across all group members (Section 4.4) o Global communication functions, which are illustrated in Figure 4.1. They include. -Broadcast from one member to all members of a group (Section 4.5) -Gather data from all group members to one member(Section 4.6) -Scatter data from one member to all members of a group (Section 4.7) -A variation on Gather where all members of the group receive the result (Section 4.8) this is shown as "allgather" in Figure 4.1. -Scatter/Gather data from all members to all members of a group (also called complete exchange or all-to-all) (Section 4.9). This is shown as "alltoall" in Figure4.1 o Global reduction operations such as sum,max,min,or user-defined functions. This includes -Reduction where the result is returned to all group members and a variation where the result is returned to one member (Section4.10) -A combined reduction and sctter operation(Section4.10.5) -Scan across all members of a group (also called prefix)(Section4.11)
Figure: Collective move functions illustrated
for a group of six processes. In each case, each row of boxes
represents data locations in one process. Thus, in the broadcast,
initially just the first process contains the item Ao, but after the
broadcast all processes contain it.
Figure 
gives a pictorial representation of the
global communication functions. All these functions (broadcast
excepted) come in two variants: the simple variant, where all
communicated items are messages of the same size, and the ``vector''
variant, where each item can be of a different size. In addition, in
the simple variant, multiple items originating from the same process
or received at the same process, are contiguous in memory; the vector
variant allows to pick the distinct items from non-contiguous
locations.
collective, vector variants
Some of these functions, such as broadcast or gather, have a single origin or a single receiving process. Such a process is called the root. root Global communication functions basically comes in three patterns:
The syntax and semantics of the MPI collective functions was designed to be consistent with point-to-point communications. collective, compatibility with point-to-point However, to keep the number of functions and their argument lists to a reasonable level of complexity, the MPI committee made collective functions more restrictive than the point-to-point functions, in several ways. One collective, restrictions restriction is that, in contrast to point-to-point communication, the amount of data sent must exactly match the amount of data specified by the receiver.
A major simplification is that collective functions come in blocking
versions only. Though a standing joke at committee meetings concerned
the ``non-blocking barrier,'' such functions can be quite useful
and may be included in a future version of MPI.
collective, and blocking semanticsblocking
Collective functions do not use a tag argument. Thus, within each intragroup communication domain, collective calls are matched strictly according to the order of execution. tagcollective, and message tag
A final simplification of collective functions concerns modes. Collective collective, and modesmodes functions come in only one mode, and this mode may be regarded as analogous to the standard mode of point-to-point. Specifically, the semantics are as follows. A collective function (on a given process) can return as soon as its participation in the overall communication is complete. As usual, the completion indicates that the caller is now free to access and modify locations in the communication buffer(s). It does not indicate that other processes have completed, or even started, the operation. Thus, a collective communication may, or may not, have the effect of synchronizing all calling processes. The barrier, of course, is the exception to this statement.
This choice of semantics was made so as to allow a variety of implementations.
The user of MPI must keep these issues in mind. For example, even though a particular implementation of MPI may provide a broadcast with the side-effect of synchronization (the standard allows this), the standard does not require this, and hence, any program that relies on the synchronization will be non-portable. On the other hand, a correct and portable program must allow a collective function to be synchronizing. Though one should not rely on synchronization side-effects, one must program so as to allow for it. portabilitycorrectness collective, and portabilitycollective, and correctness
Though these issues and statements may seem unusually obscure, they are merely a consequence of the desire of MPI to: