Automatically generate dom types

[fdietze]

Just found this: https://github.com/mdn/data

I think it should be possible to extract a huge amount of code automatically and have a more up-to-date library in general.

There is also https://github.com/mdn/browser-compat-data and https://github.com/Fyrd/caniuse

[busti]

What tools would you suggest to write a parser for that data, that generates scala code?
At work we have a script that parses our database definitions to 3 different target languages and it is a hot mess.
I would like this not to be a hot mess...
Also, when the approach works out well, this could as well be used for other targets, like scala-js-dom for example.

[fdietze]

I'm actually unexperienced with automatically generating scala-code. But here are my thoughts on how I would approach it:

Trigger code-generation with sbt: https://www.scala-sbt.org/1.x/docs/Howto-Generating-Files.html
And point the generator to a specific commit of the other repo to make builds reproducible.

The generator would look like this:
There are schemas, like https://github.com/mdn/data/blob/master/css/readme.md
Without looking into them, I guess, that we can construct an ADT, which can be used to parse all data using a json-library like circe. Then somehow transform the ADT into Scala-code.

[raquo]

I feel like our trait files are simple enough that you don't even need Scala-aware code generators to generate them. A simple Scala or JS script that concatenates strings will be much easier to understand and contribute to. CompileTest will pick up any resulting syntax / type errors anyway.

My biggest concern with generated code is the rigidity of it. If we're generating whole files, we won't be able to edit them manually. MDN data also doesn't have Scala-specific stuff such as

• Scala attribute/tag/etc names
• trait names and knowing which vals go into which trait
• trait type params including our special event types
• codecs, etc.

So this generation needs to be customizable enough to account for all those factors, as well as other general stuff like

• Somehow specifying which part of the docs needs to be put in a comment
• the ability to include or exclude some attrs/tags/etc.

In all, this seems like a big investment in time and complexity that will only save time when adding props/attributes/events, as existing ones almost never change. It will probably never pay off.

I think one way to minimize the total effort and complexity would be to instead write a less involved helper script that would read MDN data and simply print out code like this:

/** comment for attr 1 */
val attr1 = attr("attr1", ...)

/** comment for attr 1 */
val attr2 = attr("attr2", ...)

...

for a desirable subset of attrs (or props or whatever). Then you'd just copy-paste it into the right trait, trimming down the comments and doing any other manual work that the script did not account for. This would reduce the chance of errors when adding new defs and will probably be faster than doing it manually like we do now, but also faster than meticulously configuring a super advanced code generator that declaratively defines the entirety of the generated files.

[fdietze]

I feel like our trait files are simple enough that you don't even need Scala-aware code generators to generate them. A simple Scala or JS script that concatenates strings will be much easier to understand and contribute to. CompileTest will pick up any resulting syntax / type errors anyway.

Agreed.

My biggest concern with generated code is the rigidity of it. If we're generating whole files, we won't be able to edit them manually. MDN data also doesn't have Scala-specific stuff such as

• Scala attribute/tag/etc names
• trait names and knowing which vals go into which trait
• trait type params including our special event types
• codecs, etc.

So this generation needs to be customizable enough to account for all those factors, as well as other general stuff like

• Somehow specifying which part of the docs needs to be put in a comment
• the ability to include or exclude some attrs/tags/etc.

In all, this seems like a big investment in time and complexity that will only save time when adding props/attributes/events, as existing ones almost never change. It will probably never pay off.

Yes, it is a big time investment and trade-off, like with all automation tasks. But the benefits are: correctness, completeness, up-to-date for code and documentation.

I think one way to minimize the total effort and complexity would be to instead write a less involved helper script that would read MDN data and simply print out code like this:

/** comment for attr 1 */
val attr1 = attr("attr1", ...)

/** comment for attr 1 */
val attr2 = attr("attr2", ...)

...

for a desirable subset of attrs (or props or whatever). Then you'd just copy-paste it into the right trait, trimming down the comments and doing any other manual work that the script did not account for. This would reduce the chance of errors when adding new defs and will probably be faster than doing it manually like we do now, but also faster than meticulously configuring a super advanced code generator that declaratively defines the entirety of the generated files.

I agree. Before attempting to write a full-blown solution from the start, we can have simple solutions that already help a lot. I did something similar in my font-awesome facade:
https://github.com/fdietze/scala-js-fontawesome/blob/master/src/main/scala/FreeRegular.scala#L9