The NHSE progress report for Feb 1-16 follows below. Please send comments, questions, or corrections to browne@cs.utk.edu. The next NHSE progress report will be sent out on March 3, or shortly thereafter. Please send contributions to browne@cs.utk.edu before that date. Shirley Browne ************************************************************************** Shirley Browne Computer Science Dept. 107 Ayres Hall browne@cs.utk.edu (615) 974-5886 University of Tennessee Research Associate Fax (615) 974-8296 Knoxville, TN 37996-1301 Netlib Development Group ************************************************************************** ------------------------------------------------------------- NHSE Progress Report, 2/1/95-2/16/95 From Ken Kennedy at Rice (received 2/14/95) ------------------------ NHSE Review Process Jack Dongarra Geoffrey Fox Ken Kennedy Jim Pool Rick Stevens Center for Research on Parallel Computation Introduction This document describes the process by which software can achieve the "Reviewed" status within the National HPCC Software Exchange (NHSE). Currently three levels of software are recognized in the NHSE: Unreviewed. The submission has not been reviewed by the NHSE for conformance with software guidelines. This classification is for unreviewed software available on an ``as is'' basis. Partially reviewed. The submission has undergone a partial NHSE review to verify conformance with the scope, completeness, documentation, and construction guidelines. These particular guidelines are those that can be verified through a visual inspection of the submission. Reviewed. The submission has undergone a complete NHSE review to verify conformance with all the software guidelines. This classification requires peer-review testing of the submitted software. Procedure To be accorded the reviewed status, the software must first have been accorded the partially reviewed status. This precondition ensures that reviewers will be able to access all the information needed to carry out the review over the National Information Infrastructure. The author of partially reviewed software may submit his software for full review by filling out and submitting the special application form that will be reachable from the NHSE home page. On that form, the user will be asked to select a submission category from the list below: Compiler Technologies Software Tools Portable Languages Systems Technologies Visualization and Virtual Environments National Information Infrastructure Linear Algebra Numerical Optimization Numerical Simulation Algorithms and Technologies Science and Engineering Applications This will ensure that the paper is routed to the correct NHSE associate editor. The software will be reviewed according to the following criteria: Documentation. The software contains complete, understandable, correct documentation on its use. Correctness. The software is relatively bug-free and works as advertised on all provided data sets and on data sets provided by the reviewer according to the documentation.. Soundness. The methods employed by the software are sound for solving the problem it is designed for, as described in the documentation. Usability. The software has an understandable user interface and is easy to use at the level of a typical NHSE client. Efficiency. The software runs fast enough, in that slow speed does not make it an ineffective tool. Once the software is reviewed, one of two things happens. If it is not accepted, the author will be so informed and anonymous copies of the review or reviews will be provided. If it is accepted, the author will be shown a review abstract summarizing the reviewer comments. This abstract will be available to anyone who accesses the software through the NHSE. If the author finds the abstract unacceptable, he or she may withdraw the software and resubmit it for review at a later date. Editorial Board The review process will be conducted by the editorial board, chaired by Ken Kennedy: Mani Chandy John Dennis Jack Dongarra Geoffrey Fox Eric Grosse Ken Kennedy Paul Messina Cherri Pancake Horst Simon Guy Steele Rick Stevens Once a paper is submitted, it will be assigned by a semi-automatic procedure to an associate editor (one of the editorial board members). That editor will recruit one or more reviewers to review the software according to the criteria from the last section. When the review(s) are returned, the associate editor will make the final decision about whether to accept the software and inform the author of the decision. If the software is accepted, the assigned associate editor will prepare the review abstract for use by the NHSE. Procedures for Review To qualify for review, an author must provide sample data and the output or a description of results from each sample. Each reviewer will be asked to read the software documentation and try the software on some of the data sets provided by the author. In addition, it is recommended that a reviewer test the software on inputs not provided by the author. If source is available, a reviewer may wish to examine the source to ensure that the methods and programming methodology are of acceptable quality. Each reviewer will prepare all comments in electronic form and return these, along with a recommendation to the associate editor in charge of the review. Work at Argonne --------------- As part of the research in enabling technologies for the NHSE, Argonne is building a toolkit for exploring advanced web resource management technologies. Our toolkit will support hunting and gathering web pages (http, ftp, gopher), compression, indexing, transaction monitoring, parallel search and a rich language environment for developing agents. The focus of the toolkit is on discovery, distribution and managment of software (code) assets as opposed to text, images or video. For more information and to try it out, go to http://nhse.mcs.anl.gov/index.html Work at UT ---------- The article on the NHSE we submitted to IEEE Computational Science and Engineering has been accepted for publication. We will send in the final version by the end of February. I have been having trouble updating the NHSE search interface. The 1.1.beta Harvest gatherer took about two weeks to collect 120,000 HTML pages (this was a breadth first traversal three levels deep from approximately 1200 RootNodes the NHSE currently points to). The gatherer crashed a couple of times and had to be restarted. Then it misplaced most of the files somehow so that when I collect from the gatherer (either from the Broker or using command-line gather), I only get around 20,000 objects. Also, the new version of Glimpse that the Harvest folks claim is almost as fast as WAIS is still pretty slow. You can try it out yourself at http://www.netlib.org/Harvest/brokers/nhse/. I have informed the Harvest developers of these problems. (S. Browne, 2/16/95) The Argonne Web searcher described above is now linked to from the old NHSE search interface page at http://www.netlib.org/utk/brokers/NHSE/query.html. We have put up drafts of HTML forms for - users to submit user profiles - http://www.netlib.org/nse/user_profile.html (The CS&E article mentioned above asks users to fill out such a profile. The purpose of getting profiles is to help us design a better searching and browsing interface by using the set of profiles as test queries. Also in general to help us get a better idea of who our users are and of their needs and perspectives). - contributors to submit software - http://www.netlib.org/nse/software_submit.html (This form has changed since it was originally announced to some people on this list. I added a unique identifier for submitted software called a LIFN, and I added a form for changing a submission. Submitting software for full review is a special case of changing -- namely a change in requested level from "Partially reviewed" to "Reviewed". -- S. Browne) Please take a look at these forms and send comments to browne@cs.utk.edu. The survey of software currently pointed to by the NHSE has been completed (well, sort of, it's a never-ending job) and is now available online at http://www.netlib.org/nse/sw_survey.html. (The name Quentin Browne at the bottom of each page in the HTML version appears to be a case of mistaken identity by the latex2html converter). This online version will be undergoing changes and improvements during the next few days. Authors of software included in this survey will be notified and asked to formally submit their software using the above-mentioned form. Most of the work of providing cataloguing information has been done for them, but they will be asked to correct any mistakes we may have made and to fill in missing information.