adapter-static bug in handling binary body in standalone endpoints

[trenta3]

Describe the bug

According to the official documentation, Standalone endpoints can return Uint8Array types.
The given binary stream is treated correctly by adapterAuto but not by adapterStatic.

With an endpoint as simple as

// src/routes/binary.dat.js
export async function get() {
    return { body: new Uint8Array([0xB0, 0xD8]) }
}

the returned file hexdump is correctly b0 d8 when adapterAuto is used, and incorrectly ef bf bd ef bf bd when adapterStatic is used.

The sequence ef bf bd is unicode replacement character, which makes me suspect that the returned bytearray is incorrectly treated by adapterStatic as an UTF8 string or casted to a string.

Reproduction

Repository with source code: https://github.com/trenta3/bug-sk-adapter-static
Diff with respect to the standard initialization template: trenta3/bug-sk-adapter-static@96504ae

Logs

# When running in development mode
$ npm run dev
# In another terminal
$ curl http://localhost:3000/binary.dat 2>/dev/null | hexdump -C
00000000  b0 d8                                             |..|
00000002

# When running in building mode
$ npm run build
$ hexdump -C build/binary.bat
00000000  ef bf bd ef bf bd                                 |......|
00000006

System Info

System:
    OS: Linux 5.17 Fedora Linux 35 (Workstation Edition)
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 7.81 GB / 15.47 GB
    Container: Yes
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 16.14.0 - /usr/bin/node
    npm: 8.5.5 - ~/bin/npm
  Browsers:
    Chrome: 101.0.4951.64
    Firefox: 100.0
  npmPackages:
    @sveltejs/adapter-auto: next => 1.0.0-next.48 
    @sveltejs/adapter-static: ^1.0.0-next.32 => 1.0.0-next.32 
    @sveltejs/kit: next => 1.0.0-next.345 
    svelte: ^3.44.0 => 3.48.0 

Severity

serious, but I can work around it

Additional Information

No response

This behaviour has been bugging me too, so I decided to dig into the code of SvelteKit.

It seems like "pages" can only be UTF-8 encoded strings (aka what gets returned by Response.text()), while "dependencies" can be Uint8Arrays as long as the body is not defined:

kit/packages/kit/src/vite/build/prerender/prerender.js

Lines 130 to 159 in d828ca4

	const dependencies = new Map();
	const response = await server.respond(new Request(`http://sveltekit-prerender${encoded}`), {
	getClientAddress,
	prerendering: {
	dependencies
	}
	});
	const text = await response.text();
	save('pages', response, text, decoded, encoded, referrer, 'linked');
	for (const [dependency_path, result] of dependencies) {
	// this seems circuitous, but using new URL allows us to not care
	// whether dependency_path is encoded or not
	const encoded_dependency_path = new URL(dependency_path, 'http://localhost').pathname;
	const decoded_dependency_path = decodeURI(encoded_dependency_path);
	const body = result.body ?? new Uint8Array(await result.response.arrayBuffer());
	save(
	'dependencies',
	result.response,
	body,
	decoded_dependency_path,
	encoded_dependency_path,
	decoded,
	'fetched'
	);
	}

Now that raised two questions for me: What are dependencies and how do you return content without returning a body?

After reading through the documentation again I understood that Uint8Arrays are meant to be returned instead of the an object (which seemed strange to me because it made setting headers/status impossible, but then I remembered that this issue doesn't exist in server environments):

Standalone endpoints have slightly more flexibility over the returned body type — in addition to objects, they can return a Uint8Array.

I obviously couldn't confirm my theory since I couldn't figure out what "dependencies" are.

What I've found out about dependencies is that save() saves them in .svelte-kit/output/prerendered/dependencies (which doesn't even exist in my demo repository) instead of .svelte-kit/output/prerendered/pages. It seems like server.respond() populates the dependencies Map, but I couldn't confirm this by looking at the code.

Now how do we fix this?
Making "pages" Uint8Array compatible by changing

const text = await response.text()

to

const text = response.body

seems possible, but I'm in no way certain that this wouldn't break other things.

But most importantly the documentation should be more precise in what can and can not be returned by an endpoint (a dedicated type instead of { status, headers, body } seems plausible). It would be also be important to learn what the difference between a "page" and a "dependency" is and why only dependencies can return non-text content. @Conduitry could you possibly get someone on-board to clear things up?

[gbkwiatt]

Having same issue with adapterStatic

When returning Buffer in body, it corrupts all images when saved. Files are not empty and looks like are correct, but there must be some differences in binary that makes them not working. Is there any workaround for that ?

const data = Buffer.from(sourceResponse.data, 'base64');
const headers = { 'Content-Type': sourceResponse.type };

        // Return the image with original headers.
return {
            status: 200,
            headers,
            body: data,
        };