Improve template-compiler API for use in embroider

[ef4]

I'm opening this to discuss a solution to an Ember & Embroider integration issue. I don't think it rises to the level of an RFC because this is an extremely low-level API more akin to the boundary between Ember and Glimmer than to user-facing public API.

Embroider currently does hacks to get access to the features inside ember-template-compiler.js that it needs.

There are two main issues:

1. The template compiler uses shared module state, so any time I want to reconfigure it I need to evade the require cache and eval everything all over again. This makes embroider's tests slower, and it also impacts real app builds because each classic addon with custom AST transforms gets its own separately-configured template compiler instance in Embroider's stage1.

2. Embroider needs to be able to run all configured AST transforms but then print back to HBS. This lets us quarantine custom AST transforms to stage1 only, and have a clean standard template compiler in stage3. We currently do this by getting access to:

• import { preprocess } from '@glimmer/syntax' to parse (string) => AST
• import { defaultOptions, registerPlugin } from 'ember-template-compiler/lib/system/compile-options' to ensure we're calling preprocess with exactly the right arguments to achieve full compatibility with more traditional ember builds.
• import { print } from '@glimmer/syntax' to print (AST) => string

If I could instead have my dream API, it would fix problem 1 by exposing a constructor for making new compiler instances, each with its own independent configuration. And it would fix problem 2 by giving the instances an interface like:

interface TemplateCompiler {
  parse(moduleName: string, templateContents: string): AST;
  print(AST): string;
  runTransforms(moduleName: string, AST): AST;
}

While we're at it, we might consider shipping ember-source with unbundled modules for the template compiler, in addition to the bundled one. That would be a nice way to add new interface without altering the existing interface. And we could include the d.ts files.

[rwjblue]

The template compiler uses shared module state, so any time I want to reconfigure it I need to evade the require cache and eval everything all over again. This makes embroider's tests slower, and it also impacts real app builds because each classic addon with custom AST transforms gets its own separately-configured template compiler instance in Embroider's stage1.

You don't have to avoid require caches if you avoid calling registerPlugin and instead pass the plugins to precompile (ember-cli/ember-cli-htmlbars#130 implements for ember-cli-htmlbars).

[chancancode]

Embroider needs to be able to run all configured AST transforms but then print back to HBS. This lets us quarantine custom AST transforms to stage1 only, and have a clean standard template compiler in stage3.

I have doubts that this would work. Doesn’t some of the AST transform intentionally produce things that wouldn’t otherwise be possible in the actual syntax?

[rwjblue]

import { preprocess } from '@glimmer/syntax' to parse (string) => AST
import { defaultOptions, registerPlugin } from 'ember-template-compiler/lib/system/compile-options' to ensure we're calling preprocess with exactly the right arguments to achieve full compatibility with more traditional ember builds.

AFAICT, these aren't needed. See this gist for an alternative mechanism.

[ef4]

I have doubts that this would work. Doesn’t some of the AST transform intentionally produce things that wouldn’t otherwise be possible in the actual syntax?

Ah, so I should have clarified one point, which is that I definitely don't want to run the internal built-in transforms. Just the user-provided ones. That is actually another area of hax.

Presumably user-provided AST transforms target user-facing syntax.

[rwjblue]

FWIW, #19429 deprecates the APIs that do the global state mutation (and will allow us to stop the cache busting once completely culled).