Subject: more on threads

From: Ian Glendinning <[email-masked]>
Date: Sun, 18 Jul 93 22:19:12 BST

> From: Tony Skjellum <[email-masked]>

> Thanks for your responses.  I am trying to figure out how to deal with
> the creation of communicators and contexts in a way that is "thread safe."
> I will respond specifically to your two mails as soon as I can.  For now...

Hmmm, I guess my message dealt mainly with using contexts once they'd been
created.  Also, having thought a bit more about it, maybe a bit more needs to
be said about what "thread safe" means, than I suggested.  Lets take
mpi_contexts_alloc() as an example.  How I'd expect this to be used normally
would be for it to be called by one thread per process in the group doing the
allocating.  But what happens if, say, if a different thread in one of those
processes `simultaneously' makes a call to mpi_contexts_alloc()?  This could
happen if for example one of the processes in the group also belonged to
another group, and wanted to allocate some contexts for use in it, using a
different thread to make the call to mpi_contexts_alloc().  Presumably in
such a case "thread safe" means that the behaviour of the system is as if
the calls in the two threads were executed sequentially in some order.  In
other words they have to be treated as atomic operations.  I guess that
since two events are rarely simultaneous, the situation we are considering
is when one thread calls mpi_contexts_alloc and begins to execute it, but
then another thread also calls it, before the first invocation is complete.
What is required is that there is some form of locking, so that the first
invocation executes completely before the second one starts.  Of course all
of this does not prevent the example with the overlapping groups from being
erroneous, since although the allocations are atomic, they may well occur in
the wrong order.  It would be up to the programmer to make sure one thread
didn't make the call till the other had finished.  How best to do that
without introducing special thread support such as barriers to synchronize
them?  I'm not sure.  A use for sending messages to yourself perhaps?  The
same issues apply to all of the collective operations, so for example I'd
expect calls to mpi_barrier() to be made by one thread on each process in
the group to be synchronized.

> Ian asks about "contexts allocated on a per process basis."  I have no
> idea what this means.  :-)

Now I come to think of it, neither do I :-)  In the light of the above
discussion about making calls to mpi_contexts_alloc() an atomic operation,
what I said in my previous message about contexts being passed to threads by
their creators is not the only way of doing things.  The threads could
indeed create their own contexts, although there is that problem of
interleaving...  I suspect I oughtn't to mention this, but I suppose that it
could be resolved if groups were objects with globally unique identifiers
that were passed as part of the message header.  Nope?  Thought not...

> Know that I am personally in favor of having threads, I am just trying

I'm pleased to hear that.

> to reveal to everyone the complexity that is being incorporated by
> supporting the multiple threads, and that we agree specifically on what
> that complexity shall be.  Otherwise, I keep missing the mark with the
> semantics of the context chapter :-)

Yes, we should come to some agreement.  As I say though, I don't think this
is specifically a problem that affects the work of the context subcommittee,
but rather that of point to point and collective communications.
   Ian

---

From: Ian Glendinning <[email-masked]>
Date: Mon, 19 Jul 93 16:48:04 BST

Please excuse me, but, I feel I'm going to have to indulge in a bit of
commenting on my own comments ;-)

> From: Ian Glendinning <igl>

> them?  I'm not sure.  A use for sending messages to yourself perhaps?  The

It now occurs to me that the obvious method of doing this is for each thread
to use a separate context to allocate the new contexts (or to perform
whatever other collective operation it wants to).  That is, go back to using a
communicator as an argument to mpi_contexts_alloc() rather than just a group.
Now there must have been good reasons for the change to just a group in the
last draft, but I confess that they evade me.  Could someone explain what they
were please?  In fact thinking about it, the whole point of having contexts in
the first place is really to stop parallel threads treading on each other's
toes, in the sense that message transfers invoked by `parallel procedures'
(a la contexts chapter) continue to move around the system asynchronously
after the procedure returns.  In this sense I'm regarding the message-passing
system as a thread running in parallel with the user's code here, which in
fact it is.  However, when we move over to user-defined threads, the same
requirement exists to use contexts to protect them from each other.
   Ian

---

From: Tony Skjellum <[email-masked]>
Date: Mon, 19 Jul 93 10:59:05 CDT

>From [email-masked] Mon Jul 19 10:47:49 1993
>Received: from Walt.CS.MsState.Edu by Aurora.CS.MsState.Edu (4.1/6.0s-FWP);
>	   id AA26451; Mon, 19 Jul 93 10:47:48 CDT
>Received: from CS.UTK.EDU by Walt.CS.MsState.Edu (4.1/6.0s-FWP);
>	   id AA15070; Mon, 19 Jul 93 10:47:47 CDT
>Received: from localhost by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK)
>	id AA29074; Mon, 19 Jul 93 11:47:28 -0400
>X-Resent-To: [email-masked] ; Mon, 19 Jul 1993 11:47:27 EDT
>Errors-To: [email-masked]
>Received: from sun2.nsfnet-relay.ac.uk by CS.UTK.EDU with SMTP (5.61+IDA+UTK-930125/2.8s-UTK)
>	id AA29020; Mon, 19 Jul 93 11:46:25 -0400
>Via: uk.ac.southampton.ecs; Mon, 19 Jul 1993 16:46:15 +0100
>Via: brewery.ecs.soton.ac.uk; Mon, 19 Jul 93 16:37:49 BST
>From: Ian Glendinning <[email-masked]>
>Received: from holt.ecs.soton.ac.uk by brewery.ecs.soton.ac.uk;
>          Mon, 19 Jul 93 16:48:02 BST
>Date: Mon, 19 Jul 93 16:48:04 BST
>Message-Id: <[email-masked]>
>To: [email-masked], [email-masked]
>Subject: Re: more on threads
>Status: R
>
Quiescence as a requirement for library safety was the original mechanism
we proposed.  This meant that:
	i) no pending operations were present on a context
	ii) none would occur out of band while a library was using that
		context
	iii) mpi_contexts_alloc() & mpi_comm_make relied on quiescence
		to operate properly

At the last meeting, there was pummeling regarding the quiescence requirement,
in as much as, libraries could not guarantee their own safety (analogous
to a stack-based language giving new, automatic variables to an invocation
of a function).  Changing the semantics to group instead of communicator
in these two calls was meant to relieve the user of this worry.  We also
thereby removed the "you need a safe context to get a safe context."
If multi-threading needs this rule, then we have to go back to a more
restrictive model of computing for the non-threaded case; ie, that quiescence
property.  I believe that for the single-threaded case, the semantics
currently posed are good.  However, I need to understand how we manage
the multi-threaded case... 

I need an explanation of how the following scenario will be handled
(simple as possible)

1 group, 2 threads on the entire group, both threads call mpi_comm_make.

Using contexts to disambiguate threads sounds good to me, off hand, but
isn't that like giving threads "names."  Do we want this association?

Admittedly, I need to think more about Ian's comments.
- Tony

---

From: Ian Glendinning <[email-masked]>
Date: Mon, 19 Jul 93 18:24:01 BST

Oops, I mailed this to "pt2pt" instead of "mpi-pt2pt" by mistake.  From now
on I think I'll stick to "mpi-context", as I'm getting fed up getting two
copies of everything, so please look there if you're interested.
   Ian

----- Begin Included Message -----

---



> restrictive model of computing for the non-threaded case; ie, that quiescence

Thanks for that explanation.  All is now clear...

> the multi-threaded case... 

I don't see any cleaner way than using contexts.  If people are desperate
to have the non-quiescent allocator, perhaps we could compromise and have
one of each?  People wanting to use threads could then use the one with
the requirement for quiescence.

> isn't that like giving threads "names."  Do we want this association?

Yes it is like that, and yes we do want it, if we want to make threads useable.
And to do that, you've somehow got to identify them.  This does it in a way
that's not tied to any particular threads implementation.
   Ian

----- End Included Message -----