An operation that involves more than just a sender and a receiver is called a scoped operation . All processes that participate in a scoped operation are said to be within the operation's scope.
On a system using a linear array of processes, the only natural scope is all processes. Using a two-dimensional rectangular process grid, we have three natural scopes , as shown in table 4.1. Refer to [24] for further details.
Table 4.1: Scopes provided by a two-dimensional process grid
These groupings of processes are of particular interest to the linear algebra programmer, since distributed data decompositions of a two-dimensional array (a linear algebra matrix) tend to follow this process mapping. For instance, all of a distributed matrix row can be found on a process row.