adaptor-static no longer generate endpoint like sitemap.xml.ts and robots.txt.ts as a static file

[winston0410]

Describe the bug

adaptor-static no longer generate endpoint like sitemap.xml.ts and robots.txt.ts as a static file, which it did previously(I don't have an exact version number for that unfortunately. I have seen that behavior roughly 6 months ago?)

Reproduction

https://github.com/winston0410/adaptor-static-repo

Based on the log below, seems like a SSR bundle is generated, even though adaptor-static is used. It is generating a SSR bundle.

Logs

vite v2.9.14 building for production...
✓ 13 modules transformed.
.svelte-kit/output/client/_app/manifest.json                              1.19 KiB
.svelte-kit/output/client/_app/immutable/layout.svelte-1d07092a.js        0.53 KiB / gzip: 0.35 KiB
.svelte-kit/output/client/_app/immutable/error.svelte-cc54f29b.js         1.56 KiB / gzip: 0.74 KiB
.svelte-kit/output/client/_app/immutable/start-4bc1a949.js                23.14 KiB / gzip: 8.75 KiB
.svelte-kit/output/client/_app/immutable/pages/index.svelte-9d84caa9.js   0.80 KiB / gzip: 0.47 KiB
.svelte-kit/output/client/_app/immutable/chunks/index-bf1ed64f.js         6.85 KiB / gzip: 2.80 KiB
vite v2.9.14 building SSR bundle for production...
✓ 12 modules transformed.
Generated an empty chunk: "hooks"
.svelte-kit/output/server/manifest.json                        1.26 KiB
.svelte-kit/output/server/index.js                             73.97 KiB
.svelte-kit/output/server/entries/endpoints/robots.txt.ts.js   0.15 KiB
.svelte-kit/output/server/entries/fallbacks/layout.svelte.js   0.25 KiB
.svelte-kit/output/server/entries/fallbacks/error.svelte.js    0.73 KiB
.svelte-kit/output/server/entries/pages/index.svelte.js        0.33 KiB
.svelte-kit/output/server/immutable/chunks/index-b9efae8a.js   2.60 KiB
.svelte-kit/output/server/immutable/chunks/hooks-d259abab.js   0.00 KiB

Run npm run preview to preview your production build locally.
(node:17487) ExperimentalWarning: buffer.Blob is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

> Using @sveltejs/adapter-static
  Wrote site to "build"
  ✔ done

System Info

System:
    OS: macOS 11.4
    CPU: (8) arm64 Apple M1
    Memory: 164.91 MB / 8.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.15.1 - /opt/homebrew/opt/node@16/bin/node
    Yarn: 1.22.17 - /opt/homebrew/bin/yarn
    npm: 8.11.0 - /opt/homebrew/opt/node@16/bin/npm
  Browsers:
    Brave Browser: 103.1.40.113
    Safari: 14.1.1
  npmPackages:
    @sveltejs/adapter-cloudflare: ^1.0.0-next.25 => 1.0.0-next.26 
    @sveltejs/adapter-static: next => 1.0.0-next.35 
    @sveltejs/kit: 1.0.0-next.357 => 1.0.0-next.357 
    @sveltejs/vite-plugin-svelte: ^1.0.0-next.41 => 1.0.0-next.49 
    svelte: ^3.44.0 => 3.49.0

Severity

serious, but I can work around it

Additional Information

No response

[Rich-Harris]

I'm surprised you saw this behaviour previously — as far as I was aware it never worked automatically, unless you happened to link to the endpoint from a page. #4653 is intended to solve this problem (the PR is somewhat stale, because it depends on other work we haven't found time for yet, but you get the idea), but in the meantime you can work around it with this:

prerender: {
  default: true,
  entries: ['*', '/robots.txt', '/sitemap.xml']
}

[winston0410]

Thanks for your reply there! I have updated my configuration based on that snippets, and I have bumped into another issue, which I am not sure if it is opened somewhere else. I have two routes, /api/metadata that returns Array<MetaData>, and /sitemap.xml is fetching that route to build the sitemap.

It is running fine locally, but when I run it during build, I get the following error from the following code:

import fetch from 'node-fetch';

export const get: RequestHandler = async (event) => {
        //route for generating sitemap.xml
	try {
		const res = await fetch(
			(dev ? `http://0.0.0.0:${event.url.port}` : event.url.origin) + "/api/metadata"
	        );
	        //....
        }

J: request to http://sveltekit-prerender/api/metadata failed, reason: getaddrinfo ENOTFOUND sveltekit-prerender
    at ClientRequest.<anonymous> (file:///Users/xxx/Desktop/closesource/apps/jyut.info/.svelte-kit/output/server/chunks/sitemap.xml-4de5c574.js:11:15420)
    at ClientRequest.emit (node:events:527:28)
    at Socket.socketErrorListener (node:_http_client:454:9)
    at Socket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  erroredSysCall: 'getaddrinfo'
}

so seems like when adaptor-static is built locally, the url is just replaced with a placeholder, sveltekit-prerender, and a local server is not created to run the endpoints.

[winston0410]

Thanks for your reply there! I have updated my configuration based on that snippets, and I have bumped into another issue, which I am not sure if it is opened somewhere else. I have two routes, /api/metadata that returns Array<MetaData>, and /sitemap.xml is fetching that route to build the sitemap.

It is running fine locally, but when I run it during build, I get the following error from the following code:

import fetch from 'node-fetch';

export const get: RequestHandler = async (event) => {
        //route for generating sitemap.xml
	try {
		const res = await fetch(
			(dev ? `http://0.0.0.0:${event.url.port}` : event.url.origin) + "/api/metadata"
	        );
	        //....
        }

J: request to http://sveltekit-prerender/api/metadata failed, reason: getaddrinfo ENOTFOUND sveltekit-prerender
    at ClientRequest.<anonymous> (file:///Users/xxx/Desktop/closesource/apps/jyut.info/.svelte-kit/output/server/chunks/sitemap.xml-4de5c574.js:11:15420)
    at ClientRequest.emit (node:events:527:28)
    at Socket.socketErrorListener (node:_http_client:454:9)
    at Socket.emit (node:events:527:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  type: 'system',
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  erroredSysCall: 'getaddrinfo'
}

so seems like when adaptor-static is built locally, the url is just replaced with a placeholder, sveltekit-prerender, and a local server is not created to run the endpoints.

A follow up on the issue, as event.url will be set as a default value sveltekit-prerender when the adaptor-static is used, apart from avoiding the use of fetch to get data from another endpoints, we will have to change the implmentation for endpoints, if we want it to work with adaptor-static(e.g. introduce env var to specify the domain name). I think this behavior is against the idea of adaptor, where you keep the same code but change the deployment result by using different adaptor.

My suggestion is we allow user to set the default value for event.url in svelte.config.js, so that the implmentation of the handler doesn't need to be changed and can be universal. What do you think? @Rich-Harris

[Rich-Harris]

Ah yeah, that's right. The sveltekit-prerender thing is just because we need something to construct a valid URL with, but as you say there's no server, so fetch can't possibly work.

A potential solution would be to override fetch during prerender like so:

globalThis.fetch = (url, opts) => {
  if (url.startsWith('http://sveltekit-prerender/') {
    const request = create_request(url, opts);
    return server.respond(request); // calls the endpoint directly
  }

  return fetch(url, opts);
}

Instead of doing this sort of thing to construct the URL...

(dev ? `http://0.0.0.0:${event.url.port}` : event.url.origin) + "/api/metadata"

...you'd do this:

new URL('/api/metadata', event.url)

we will have to change the implmentation for endpoints, if we want it to work with adaptor-static(e.g. introduce env var to specify the domain name). I think this behavior is against the idea of adaptor, where you keep the same code but change the deployment result by using different adaptor.

I'm not sure I follow this, can you provide a concrete example?

[winston0410]

we will have to change the implmentation for endpoints, if we want it to work with adaptor-static(e.g. introduce env var to specify the domain name). I think this behavior is against the idea of adaptor, where you keep the same code but change the deployment result by using different adaptor.

An example would be building a sitemap for adaptor-static or an adaptor that can do SSR, like adaptor-vercel. For example, the following code would work fine for adaptor-vercel:

import { SitemapStream, streamToPromise } from "sitemap";
export const get = async (event) => {
	try {
		const sitemapStream = new SitemapStream({
			hostname: event.url.origin, // This will be the domain of the application, detected in the runtime by the serverless function
			xmlns: {
				image: true,
				news: true,
				video: true,
				xhtml: true
			}
		});

               // Add entries to the sitemap

		return {
			headers: {
				"Content-Type": "application/xml"
			},
			body: result.toString()
		};
	} catch (error) {
		console.error(error);
		return {
			status: 500
		};
	}
};

But the above implementation won't work for adaptor-static, as right now event.url.origin is set as sveltekit-prerender. The generated sitemap will be something incorrect like this:

<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1"><url><loc>http://sveltekit-prerender/</loc><lastmod>2022-07-13T00:06:53.840Z</lastmod><image:image><image:loc>http://sveltekit-prerender/</image:loc></image:image></url></urlset>

If the value of sveltekit-prerender can be changed in svelte.config.js like this, we can keep the above implementation.

//svelte.config.js
const config = {
  kit: {
    prerender: {
     default: true,
     hostname: 'foo.com' // Possible solution. The value used to replace `sveltekit-prerender`. process.env can be used here as well.
    }
  }
}

export default config