distributed memory multiprocessor

Each CPU has its own associated memory.  The CPUs are connected
in some way and may exchange data between their respective memories.
Distributed memory systems may be either SIMD or MIMD.

Distributed memory SIMD machines are also known as processor array
machines.  The processors of these machines operate in lockstep --
i.e., all processors execute the same instruction at the
same time (but on different items), and consequently no synchronization
between processors is required.  All currently available DM-SIMD
machines use a front-end control processor that issues the instruction
sequence that is executed by the processor array.  DM-SIMD machines
generally perform well on digital signal processing and image processing
and on certain types of Monte Carlo simulations where virtually
no data exchange between processors is required and exactly the
same operations are done on massive datasets.

Distributed memory MIMD (DM-MIMD) machines are the fastest growing
family of high performance computers.  
Theoretically, this class should be able to outperform the others.
However, these machines are
more difficult to use than shared-memory systems and DM-SIMD machines,
as data decomposition, task decomposition, and synchronization
must be dealt with explicitly.
DM-MIMD machines may have processors connected in a fixed topology
or the topology may be flexible.
Many of the newly introduced massively parallel DM-MIMD systems
favor a 2-D or 3-D mesh (torus) structure, the rationale being
that most large-scale physical simulations can be mapped efficiently
on this topology.

\cite{van-der-Steen-overview}