Subject: Re: t3d resutls

From: Kenneth R Koch <[email-masked]>
Date: Fri, 13 Oct 1995 08:44:40 -0600

Pat Worley writes:
> > > Ken,
> > > > Unfortunately, it appears that the T3D would pose great problems
> > > for PARKBENCH utilizing PVM3.3.8.  Hopefully, when the MPI codes are
> > > released, we'll be able to get a much easier view.  It turns out that
> > > NAS is producing their own vanilla MPI benchmark versions that we will
> > > be including within PARKBENCH--this is very fortunate.
(this was from Mike Kolatis, UTK)
> > 
> > So ParkBench is not suitable for the T3D (nor the T3E when it comes
> > out).  Will the MPI version really address the T3D porting issues we
> > have uncovered?  I don't think so unless there is a specific effort.
> > Does anyone else other than me see that not supporting the T3D (a major
> > MPP) is a major short coming of ParkBench?
(I wrote this)
> 
> ...
> 
> What is the background for this message? What is unfair about using PVM3.3.8?
> I have run PSTSWM (one of the compact application codes) with both PVM and
> SHMEM on the T3D with no problems, and SHMEM is faster. Is this the issue?
> 
> Pat Worley

It turns out that most of the Parkbench codes are not portable to the
T3D for 3 major reasons.  It would take a concerted effort on Parkbench
code contributors (or CRI) to deal with the CRI compiler and T3D specific
obstacles, but you are absolutely correct in that it can be done.  A
clean port to a C-90 would be a good logical first step.

>From an earlier mail message to Jack Dongarra, Ken Koch writes:
> LANL has some experiences and results to report for the T3D thanks to
> Sowmini Varadhan's (UTK-Berry) great efforts here this summer.  The CRI
> special Fortran-to-C linkage (especially character strings in PBLAS &
> BLACS), the T3D non-hosted PVM mode, the T3D 64bit word size, and the
> gettimeofday() system timer created problems for the ParkBench codes as
> distributed.  We were only able to run the NAS set, and we couldn't get
> CG to run at all because of insufficient info about setting parameters
> (we even called NAS).  We had tons of trouble with the ScalaPack codes,
> especially the BLACS PVM implementation and thus never got them working
> at all.  The low-level benchmarks also had PVM problems for the T3D and
> were being rewritten anyway so we didn't run them.  Finally, our
> ParkBench NAS results aren't good; some are uniformly 4 to even 20
> times slower than CRI's reported NAS results, and the rest show
> slowdown trends, not speedups!  We ran out of time before we could
> trackdown what was really happening.  Bottom line is that T3D ports of
> ParkBench are a mess right now.  And I would guess that little would
> change for the T3E.  So...

>From a very recent mail message to Jack Dongarra, Ken Koch writes:
> The T3D port now looks to be mostly CRI software and T3D specific
> problems.  And CRI is clearly the problem because of their screwy
> non-portable language and Unix support and the T3D data types.
> At this point in time, I fully support ParkBench NOT supporting this
> machine.  There were 3 major stumbling areas on the T3D:
> 	1) CRI T3D PVM version is 3.3.5 or 3.3.7 EXCEPT that
> 	   some functionality just isn't there.  Also, the T3D PVM
> 	   implementation has lots of additional restrictions on using
> 	   PVMDATAINPLACE that are definitely not standard to PVM.  So
> 	   in general you just have to back off to PVMDATARAW.
> 	2) Fortran to C linkage of character strings.  The unique CRI
> 	   way requires a special "character descriptor" data type and
> 	   some auxilirary routines to handle Fortran character strings
> 	   passed into C routines.  They use this on their entire
> 	   product line, so it isn't just T3D specific.  This is really
> 	   only a problem with the LINALG set because of it extensive
> 	   use of character string args for the PBLAS and BLACS
> 	   routines.  Those packages as provided in ParkBench won't
> 	   even work on a YMP or C-90 because of this; and I think this
> 	   is a larger issue you may want to address outside of
> 	   ParkBench.  Ps: It also isn't just an easy matter of using
> 	   CRI's provided BLACS as they have a different API!  I
> 	   thought BLACS was to be a standard; what gives?
> 	3) The T3D only supports 64 bit integers and 64 bit reals
> 	   under the standard Fortran 77 (cft77) environment.  This has
> 	   repercussions in many places, one of which is their special
> 	   INTEGER8 PVM datatype.  Now the new Fortran 90 (cf90) on the
> 	   T3D does allow 32 reals if explicitly declared real*4, but
> 	   it still defaults to 64 bit integers - nice huh?  It may
> 	   support integer*4 declarations as well, but I don't recall.
> 	   Most T3D sites won't have the F90 environment.  So this is
> 	   a thorny area.

Parkbench needs to address the SHMEM, NXLIB, MPL, ... issues.  Are they
or will they be legal substitutes for PVM.  What are the expected "run
rules"?

SHMEM is faster than PVM on the T3D, but it isn't just a drop in.  For
one, the synchronization issues are completely different with a
one-sided communication mechanism.  So you typically end up with an
#ifdef code structure and several vendors have complained that you can
NOT use CPP with Fortran code.  But if you are careful you can!  So
then you start getting a large multi-flavor support issue and Parkbench
should probably address this issue soon as well.

Maybe we can decide what to do with the T3D issues at the 11/3 meeting.
There is also a significant CM5 issue (no vector units!).

--Ken Koch, CIC-19, LANL