Shared filesystem to support multiple, concurrent workers

[adamziel]

Playground performance would greatly increase if it could spawn multiple workers to handle concurrent requests, much like php-fpm does. The largest blocker to that is having a shared filesystem with locks support. Let's find a way to support that.

Ideas:

• Implement a MEMFS variation based on a SharedArrayBuffer
• Migrate to WASMFS – is it synchronized between concurrent workers?

Related resources

• https://sqlite.org/wasm/doc/trunk/persistence.md
• PHP.wasm: Explore WASMFS/OPFS #1878

[brandonpayton]

Migrate to WASMFS – is it synchronized between concurrent workers?

My original understanding was that WASMFS was owned by a single execution thread, but from this brief summary in the Emscripten docs, that may be an incorrect understanding:

The JavaScript-based file system was originally written before pthreads were supported and when it was more optimal to write code in JS. As a result it has overhead in pthreads builds because we must proxy to the main thread where all filesystem operations are done. WasmFS, instead, is compiled to Wasm and has full multithreading support. It also aims to be more modular and extensible.

We don't use pthreads though but rather multiple, separate php-wasm instances.

We could explore using pthreads again. At the moment (since I don't know how Emscripten pthreads support works), pthreads seems more magical and opaque to me so that multiple, discrete php-wasm instances seems much simpler.

[brandonpayton]

Implement a MEMFS variation based on a SharedArrayBuffer

Today, I spent some time testing and exploring the cross-origin isolation that is required to use SharedArrayBuffer. Having SAB as a tool could unlock so many possibilities. And my mind and enthusiasm keep coming back to a binary, shared FS implementations.

Here are some messy notes and thoughts from that exploration:

@adamziel, in another conversation, I believe you mentioned trouble with YouTube embeds in Playground when cross-origin isolated (with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: "require-corp" or "credentialless". By default, it doesn't work at all, but if you embed with <iframe ... credentialless> in Chrome, it can work.

Supposedly, Firefox supports Cross-Origin-Embedder-Policy: credentialless but doesn't yet support <iframe credentialless>. I wonder if we can work around the FF limitation by using Service Worker to:

• Look up a fetch event client with Clients.get(fetchEvent.clientId).
• fetch() with mode: no-cors if client.frameType === 'nested' and client.url === fetchEvent.request.url.

Safari doesn't yet support Cross-Origin-Embedder-Policy: credentialless. Their team expressed support for the feature here, but implementation appears stalled:

This adds a lot of complexity and seems to have quite a few pre-requisites (Private Network Access, ORB, anonymous iframes). I am not convinced it is worth the effort at the moment.

🤔 Some ideas for workarounds:

• Provide placeholder content for embeds that fail within a crossOriginIsolated document. This would affect all failures to request the iframe src, not just failures due to cross-origin isolation.
• Use the CORS proxy for embeds.
  • This would require additional work in the CORS proxy:
    • Add Cross-Origin-Resource-Policy response header
    • Detect and satisfy requests for resources relative an original proxied request (e.g., /some-script.js in the HTML document returned for a youtube.com embed). Without this, a YouTube event is not properly loaded via the CORS proxy. This might be doable using the Referer header.
  • We don't really want to proxy streaming video but at least want the embeds to appear real.
  • This may not be wise... would need to think through security ramifications.

Some other notes:

• If we have single-worker and multi-worker modes, maybe both could use the same underlying FS type, with single-worker mode using an ArrayBuffer and multi-worker mode using a SharedArrayBuffer.
• Also relevant to SharedArrayBuffer: @adamziel made an emscripten-based PoC a while back. It is currently available in this branch. If we want to discuss further, we can create a PR from it.
  https://github.com/WordPress/wordpress-playground/tree/shared-filesystem