Design: thread shutdown handlers for proxied work

[tlively]

Problem: The current machinery for proxying work to threads does not allow for thread shutdown to be handled gracefully. If a message arrives after thread shutdown, there is no known way to notify the sender because we do not know at that point who the sender was. Even if we did know the identity of the sender, there would be no good way to notify them of the shutdown. We could postMessage the sender, but they may be synchronously waiting for their original message to be processed, so they may not receive the cancellation message. We couldn't notify the sender with Atomics.notify either because we wouldn't know whether they are waiting for such a notification and we don't know what address and value to use for the notification even if they were.

We currently handle this lack of shutdown notifications by calling it user error if a thread dies while work is being proxied to it. This is sufficient for systems like WasmFS that can guarantee that the proxy target will not unexpectedly die, for example because it never runs user code. This is insufficient, however, for systems that need to proxy work to arbitrary user threads, such as the dynamic loader.

Requirements: Since senders may be waiting for their messages to be processed in different ways, they need to be able to specify how they should be notified of thread shutdown. This implies that each message should provide arbitrary code to be run if the target thread starts shutting down before the message can be processed, which means that the C stack still needs to exist on the target thread when the messages' shutdown handlers are run. It should not be possible for new messages to arrive later in shutdown and remain unhandled.

Design: The following is a sketch of a design that adds graceful shutdown handling as a layer underneath the current proxying system.

1. A pointer to an em_task_queue is added to struct pthread. This task queue is the pthread's "mail box." The mail box is "closed" if the pointer is null and "open" if it points to a valid em_task_queue.

2. To send a message to a thread, atomically check that its mail box is open and enqueue a message on it. If the mail box is closed, the target thread is shutting down or has already shut down, so return an error immediately. If the enqueue succeeds, notify the thread with a postMessage via the messageRelay or with an Atomics.notify if the target thread is waiting with Atomics.waitAsync.

3. A message is the typical function pointer and void* user data and an additional function pointer that will be called to handle a shutdown that begins before the message can be processed. The normal message function pointer will always be called directly from the JS event loop, but the shutdown handler may be called with arbitrary user code on the stack and should only perform trivial work like sending a postMessage, setting a flag, or notifying a waiter.

4. On thread shutdown, the mail box is atomically closed. This ensures that no further messages will be enqueued. The thread then dequeues the messages already in its closed mail box and calls their shutdown handlers.

5. (Optional) To prevent TOCTOU bugs where messages may be proxied to threads that have already finished shutting down, pthread structs could be placed into quarantine and reused after a quarantine size threshold has been reached rather than immediately freed. This would make it possible for the proxying code to find the closed mail box and return an error immediately without ever dereferencing unallocated memory.

cc @sbc100 how does this sound? I plan to add new error handling and reporting to the proxying.h API building on top of the functionality described here because that sounds generally useful. Do you think you'd want to continue building on the proxying.h API for the dynamic loader or would you want to build directly on this lower-level API?

[sbc100]

I suppose that this mechanism would allow me to avoid the tracking finishedThreads and outstandingPromises in my current PR: #18376 ? Is that right? If so it sounds like a good idea to me. I would still have slight preference for landed what I have to today and transition to this new and more generic solution once it is in place. For one thing, it would give this new system a good test case to know that it working for a known use case before we land it.

One question about the proposed design: Does that mean each thread can only have single task queue? I though that the point of the task queues were that there could be many of them.

[tlively]

Yes, this should let you avoid finishedThreads and oustandingPromises. It will also scale better to handle thread shutdown for APIs like emscipten_proxy_sync that can't use promises because they synchronously block on a condition variable.

Landing what you have and switching to this design once it's ready sounds good to me.

For these new mail boxes, yes, each thread would have just a single queue. But this is a lower-level API layered underneath and serving a different purpose than the existing proxying.h API. The proxying.h API gives users full control over when the queue is processed and lets work be partitioned between different queues that are processed at different times. These new "mail box" queues are more like replacements for postMessage: besides on thread shutdown, they will only ever be processed when the stack is empty.

[tlively]

This is now implemented and merged.