Turbopack Share Memory

[owjs3901]

Summary

Turbopack implements a parallel loader by running it in a separate process after a certain amount of time when executing a loader.
However, this interferes with my intention to obtain global values.

I need a way to share data within the Turbopack loader, or an option to ensure that only a single process is used.

Additional information

No response

Example

No response

[haroonmaqbool]

Hi @owjs3901, turbopack runs each loader in its own worker process to enable parallel execution, which means that global variables and shared memory cannot be directly accessed across different loaders. This process isolation is intentional for performance and stability but prevents sharing in-memory state between loaders. If you need to persist or share data, the recommended approach is to use an external store such as a temporary file, a JSON cache, or a lightweight in-memory database like Redis to coordinate data between processes. However, if your goal is simply to avoid parallelization for debugging or deterministic builds, you can disable Turbopack’s parallel loader by setting the environment variable TURBOPACK_PARALLEL=0. This ensures that all loaders run in a single process. At the moment, Turbopack doesn’t provide an API to share JavaScript memory objects directly between loaders, so using an external synchronization layer or single-process mode is the most reliable solution.

[owjs3901]

Thanks for the suggestion.
I tried setting the TURBOPACK_PARALLEL environment variable, but it doesn’t seem to have any effect.
Also, I couldn’t find any reference to TURBOPACK_PARALLEL in the Next.js or Turbopack source code.
Could you clarify where this variable is used or how it’s supposed to work?

[owjs3901]

I implement feature

#85268

[bgw]

Hey @owjs3901, thanks for writing a PR, but we can't accept this for a few reasons:

• Even if the loader execution is limited to a single CPU, the process pool is ephemeral, and we may choose to spin down the worker at some later point, so it doesn't guarantee your global state will actually be retained.
• Turbopack's experimental filesystem cache means that loader execution may be persisted to disk across executions of next dev or next build, so this won't work as expected. We want to make the filesystem cache default in a future release.
• Global state in a webpack loader is a huge footgun. It's very easy to get it wrong and break caching. We know this from experience because our Babel loader does some custom hand-rolled caching of configuration objects, and we've run into serious bugs with it.
• Parallelizing loader execution is important for keeping Turbopack fast. People will inevitably blame this on us, rather than the plugin that's disabling parallelism.

We do recognize that there is a use-case for reusing work across loader invocations. We have a similar problem with Tailwind, where each worker process ends up having to rescan the filesystem for used classnames because we don't have an API for Tailwind to do that once and re-use the information.

We intend to produce some proper API for this kind of problem. That might be tailored towards Tailwind's usecase, but whatever solution we make will be usable by other tools. We do not currently have any timeline for this, as we need to do a lot of exploratory work first.

[owjs3901]

Thank you for your kind and detailed response.
I assumed there might be reasons for not providing such an option, but it’s really interesting to hear that serious bugs have occurred in the past because of this.

In my case, sharing data between loaders is essential for the project I’m working on — a zero-runtime CSS-in-JS library called Devup UI.(https://devup-ui.com)
It currently works correctly in all environments except Turbopack, so I’d love to contribute or collaborate to help make a solution for sharing state between loaders possible sooner.

Alternatively, if you have a different idea after looking at our library, I’d really appreciate any guidance on a potential approach I might have missed.

Thank you again for your time and thoughtful explanation.

[bgw]

Thanks, as I mentioned above, we don't have a timeline for this yet, and because we're not really sure what API we want yet, it likely wouldn't be a good task for an external contributor. We've also got some other architectural issues with how we run loaders that we'd probably want to fix before adding this.