Active messages were designed to be as close to the hardware model as possible. In this way, protocol processing is minimized, even when the network is somewhat unreliable. All implementations assume that a very small percentage of packets will be dropped (as a result of collisions, propagation delay, media quality, etc...). The objective is to pay a performance penalty only in the infrequent event that a packet is lost. Hence, a very simple request-and-reply protocol will suffice to get the data through reliably. This protocol will be covered later in the paper.
In theory, the AM layer has direct access to the network device's FIFOs and receives interrupts upon packet reception. On the CM5 and similar architectures where network interrupts are not available, the AM layer polls the interface for available data. Unfortunately, direct access to the network is often very difficult to achieve. In fact, many AM implementations consist of large kernel patches to make such access easier. When using an unreliable transport, additional problems must be solved. These include flow control, error detection, packet timeouts, and fragmentation. All of these problems, as well as that of portability, can be solved by using the operating system's networking protocols. These, however, incur significant overhead and thus limit the potential bandwidth. Higher performance interfaces are available. Unfortunately one must have root privileges to access them in all commercial Unix implementations.
In this paper we explore the various alternatives available in designing a portable AM layer to be incorporated into the PVM suite. We conclude with specific suggestions to assist the developer in achieving a higher network throughput.