Use pure ES modules

[ehmicky]

See background at https://github.com/netlify/team-dev/issues/36

Once #3512 is done and released, we should use pure ES modules and make a major release.

This is more than just switching from CommonJS to import/export. See this list for other changes which might be involved. This should be broken in many PRs, as much as possible, to lower the risk. Also, non-breaking changes (such as adding file extensions in imports, or loading JSON files differently) should be done before the breaking changes (such as using import/export statements).

Since we can make a major release, upgrading would require a user action and no sites would be broken.

[ehmicky]

This was discussed in our guild sync and @lukasholzer decided to start moving on this item.

[ehmicky]

After discussing with @lukasholzer and @eduardoboucas, we decided on the following plan:

• 1. Adding .ts file extensions to all files and import/export. Without types yet, nor additional tooling (except for tsc) or re-architecturing.
• 2. Move to pure ES modules.
• 3. Complete the TypeScript migration, by adding types, etc.

[lukasholzer]

As going further with the migration it seems that we are blocked by oclif therefore I suggest implementing this issue earlier: https://github.com/netlify/pod-workflow/issues/343 (internal issue)

[lukasholzer]

@ehmicky, @erezrokah I hit some further roadblocks on the cli repository I wanted to discuss with the group.

The CLI is using the require cache mechanism to serve a clean state of the functions locally when developing. When we switch to es modules this feature is not available anymore.

So I'm asking you about ideas on how we can solve that. Some options came to my mind.

1. Migrate to dynamic imports without any cache clearing just plain imports. (less preferred)
2. Always bundle locally with esbuild to have the dependencies inside the file, make esbuild default and use it's serve mode. (No problem with dependency caching at all)

Please add your opinions on that one. cc @eduardoboucas

[eduardoboucas]

Always bundle locally with esbuild to have the dependencies inside the file, make esbuild default and use it's serve mode. (No problem with dependency caching at all)

We should do everything we can to make sure the local development experience mimics the production environment as much as possible. esbuild is not the default bundler in production, and it's unlikely it will be anytime soon, so we shouldn't use it in the CLI unless people opt into it.

I know that we had some issues in the past with the cache mechanism when serving functions, with paths that should be updated after a file save but weren't.

If this functionality is only available in CJS, can we have a CJS file that's included by the remaining ES codebase? And if that's not an option, will that not be a problem when we load CJS functions defined by users?

[ehmicky]

When it comes to mimicking require.cache with pure ES modules, I believe dynamic imports can be used with a cache busting query parameter.

With CommonJS:

require('/path/to/example.js') // Cache not used
require('/path/to/example.js') // Cache used
delete require.cache['/path/to/example.js']
require('/path/to/example.js') // Cache not used

With pure ES modules (note: this only works if the importing file is using pure ES modules):

await import('/path/to/example.js?one') // Cache not used
await import('/path/to/example.js?one') // Cache used
await import('/path/to/example.js?two') // Cache not used

Please let me know if my comment is out of topic, as I do not know how caching is currently implemented with functions served locally.