Selecting the appropriate software for a high-performance computing task is difficult. Packages differ in capabilities, features, and quality. Comparative evaluations, when they are available, usually come from the author of one of the packages. As a result, comprehensive, independent, and unbiased evaluations are not normally readily available, despite the obvious value such information would be to users.
The National HPCC Software Exchange (NHSE
), a Center for Research on Parallel Computation
(CRPC) project for the collection, distribution, and evaluation of
software and information produced by HPCC programs, is currently
undertaking comparative evaluations of high-performance computing
software with a view to satisfying this need. Our goal is to provide
independent, unbiased comparative evaluations of HPC software of wide
applicability. Users get easy access to side-by-side comparative
evaluations based on consistent and objective.
Our current evaluation focus is on the Parallel Tools Library (PTLIB), a new software repository for parallel systems software and tools, and HPC-Netlib, a high performance branch of the Netlib mathematical software repository. We refine a high-level evaluation criteria to the domains in these two areas and for each package in a particular domain, we apply a consistent set of criteria to assess various characteristics of the software. The evaluations, as well as author and user feedback, are made available via the Web.
Although the software evaluation part of NHSE activities is still in the early stages, many packages have already been evaluated. However, the evaluations will be on ongoing task. Our evaluation criteria and procedures may evolve as software pool grows and as we gather comments from software authors and users.
Section 2 describes our approach to evaluating high-performance computing software in more detail. Sections 3 and 4 describe the evaluation criteria and current status for our evaluations of software in PTLIB and HPC-Netlib, and Section 5 summarizes our results.