Make it possible to return more than JSON from server-only `load` functions

[Rich-Harris]

Describe the problem

Right now, the output of load in +layout.server.js or +page.server.js has to be serialized with JSON.stringify so that it can be transported to the client.

Occasionally it's useful to transport values that can't be serialized as JSON. Date springs to mind, along with things like this, where anything expecting selected to be a member of options post-deserialization will be disappointed:

export function load() {
  const options = [
    { flavour: 'chocolate' },
    { flavour: 'strawberry' },
    { flavour: 'banana' }
  ];

  const selected = options[0];

  return { options, selected };
}

Describe the proposed solution

We have a proven technique for this: https://github.com/rich-harris/devalue. It's fast, and doesn't involve a deserialization step. (It doesn't currently support BigInt though, we'd need to add that.)

To work with CSP, we can't eval the result (including by creating a <script>), we need to import it. Since we need to get fresh data when navigating back to a previously-visited page, we need to use a cachebusting mechanism; since modules live in a cache that we have no way of purging, we need to be careful to avoid memory leaks.

We can't therefore do this...

export const data = {...};

...and must instead do this, so that the data can be garbage collected once it's been used:

window.__data = {...};

On the client side, it would look something like this:

let uid = 1;

// later
await import(`${url.pathname}/__data.js`);
const data = window.__data;
delete window.__data;

Alternatives considered

There's a variety of alternatives to devalue listed here: https://github.com/rich-harris/devalue#see-also, though I'm not aware of any reasons to favour them.

Or, we could just stick to JSON, which has better performance characteristics if you have a huge amount of data.

Importance

nice to have

Additional Information

No response

[Conduitry]

Is it sensible to make it a build time configuration concern whether to use JSON or devalue for a given page? (We obviously don't want to be doing this analysis at runtime and the client would have no way of knowing the result of this analysis ahead of time.) Explicitly opting in to the more performant JSON might be nice.

alternatives to devalue listed here: https://github.com/rich-harris/devalue#see-also, though I'm not aware of any reasons to favour them.

@Rich-Harris superjson supports ObjectId from Mongo/BSON

So, where can I can change now the serializer to superjson myself? See also #6194

My last comment was on-topic. devalue doesn't support ObjectIds and they get blocked despite passing a JSON.stringify({ a: new ObjectId() }). The deserialization would need an new ObjectId(...) but even without (just being a string) it'd be still fully reversible.

[chanced]

Currently, check_serializability doesn't check to see if an object has a toJSON method. This would solve serialization of Date and @2foo's ObjectId issue.

What I'm not sure of is if there's a reason for that. Is this due to concerns over deserialization or typings?

[Conduitry]

We're currently specifically not checking for .toJSON(), because things serialized that way would deserialize (JSON.parse) differently in the browser. You would have different data in the component depending on whether it was rendered on the server or in the browser. See e.g. #6093.

[chanced]

That makes sense. While a bit of a performance hit, would it not make sense to run serverside through a serde pass-through?

If so, the need for check_serializability would presumably be diminished.

edit: I guess this could cause an issue with typings.

or (again) why can't we specify out own serializer, e.g. superjson which supports all extra types and recipes for stuff like Prisma.Decimal + ObjectId and call it a day? why do you want to reinvent the wheel?

[Conduitry]

The current plan is to use devalue, a library that already exists, and which superjson's readme cites as prior art, and which has better performance characteristics than superjson.