We propose a prototype library of Basic Linear Algebra Parallel I/O Subprograms (BLAPIOS) for dense matrices. As discussed in Section 3, we would like the BLAPIOS to be compatible with any future standard for parallel I/O that emerges. Thus, we describe only the high-level functionality of the BLAPIOS, and defer specifying the detailed semantics and syntax. A similar approach has been taken by Toledo and Gustavson in the Matrix Input-Output Subroutines (MIOS) which forms part of the SOLAR library for out-of-core dense matrix computations [15].
Before describing the BLAPIOS we shall consider
the fundamental I/O operation supported by the BLAPIOS in which a
rectangular array of data is read from (written to) the out-of-core file into
(from) a given in-core array. Suppose the data in the out-of-core file
and the in-core array are represented by the index ranges
, and (i:i+m-1,j+n-1), respectively, as
shown in Figure 5. As in the PBLAS and ScaLAPACK libraries,
submatrices are regarded as global entities and are
referenced by global indices.
Figure: Fundamental I/O operation for matrices.
For a shared file the indices k and 
can refer to any element in the
out-of-core file. However, for a distributed file the submatrix referenced
in the out-of-core file must have the same data distribution as that in
the in-core array. This is because both the out-of-core distributed file
and the in-core array are distributed data objects.
An example of compatible and incompatible data distributions for a
distributed file and an in-core matrix are shown in Figure 6.
Figure: On the left we show two submatrices of a distributed file. On
the right is an in-core array. Both the distributed file and the in-core
array are distributed over a 
mesh of processes. The
smaller squares represent 
blocks of elements.
The distribution of the submatrix in the lefthand distributed file is
compatible with that in the in-core array, while the distribution of
the submatrix in the righthand distributed file is not.
The routines comprising the BLAPIOS library are arranged in three groups.