Usage statistics on optional chaining in CoffeeScript

[alangpierce]

Hi! I've been following a lot of the discussions here, and to help inform these discussions, I wrote a tool that gets statistics of how the ?. operator is used in practice in real-world CoffeeScript projects. CoffeeScript includes basically all of the features under discussion (short circuiting, soaked new, soaked delete, soaked assignments, parens to block short-circuiting), so I think it's a good case study to see how these details play out in the real world.

(For a little more background about me, I've been the main person working on the decaffeinate project for quite a while now, and I implemented ?. and similar operations in decaffeinate, so I've worked with the nitty-gritty details of these operators quite a bit. My personal preference for JS optional chaining is to keep things simple; I was unpleasantly surprised when I learned how magical ?. is in CoffeeScript. But I'm a little biased because I've generally viewed this stuff from an implementer's perspective.)

Here's the repo: https://github.com/alangpierce/coffeescript-soak-stats

The README has a detailed explanation of the different stats when run on about 500,000 lines of "typical" code found on GitHub (Atom, ShareLaTeX, CodeCombat, SwitchyOmega, Trix, Vimium, YakYak, and Bacon.js). Here are the results:

Total files: 2489
Total lines: 475208
Total soak operations: 4627
Total soaked member accesses: 3811
Total soaked dynamic member accesses: 240
Total soaked function applications: 576
Total soaked new invocations: 0
Total soak operations using short-circuiting: 1522
Total soak operations using short-circuiting (excluding methods): 233
Total soaked assignments (including compound assignments): 37
Total soaked deletes: 1
Total cases where parens affected the soak container: 0
Total soak operations chained on top of another soak: 564

Some observations:

• Soaked new was never used and soaked delete was only used once.
• Even though parens can be used to block short-circuiting, e.g. (a?.b).c, it never came up in practice (probably because it would just make the code crash).
• Soaked assignments (like a?.b = c) had some usages, although of course they were much more rare than other uses of ?..
• Short-circuiting wasn't used in 67% of use cases. If short-circuiting only applied to direct method syntax like a?.b(), that would cover 93% of all use cases.
• It was about twice as common for people to write chains like a?.b?.c (which don't rely on short-circuiting) than a?.b.c (which rely on short-circuiting).

So I think this is at least an argument for leaving out new, delete, and meaningful parens, unless they somehow makes things simpler.

Happy to dig into the specific examples for these or run other statistics if people want. And of course it's open source and published on npm, so feel free to hack on it or try it on your own CoffeeScript code. I was reasonably careful and there are tests, but it's still possible some of these numbers are buggy.

[alangpierce]

One thing to keep in mind when interpreting these numbers is that "number of usages" probably isn't the best way to evaluate how important/useful a feature is, it's just the easiest thing to measure. Still, I think it's a rough proxy for usefulness. Ideally, you should also weight by how much of an improvement the syntax is over the alternative. Simple cases like a?.b (with no chaining) are common, but also only a minor syntactic improvement over the common alternative a && a.b. Cases like a?.b?.c?.d?.e are rare, but are probably more important because they're a major improvement over the alternative in JS.

[claudepache]

@alangpierce Thanks for the analysis.

Soaked assignments (like a?.b = c) had some usages, although of course they were much more rare than other uses of ?..

I wonder what are examples of such usages?

[alangpierce]

@claudepache here's a gist with all 37 examples: https://gist.github.com/alangpierce/34b7aa40cda51b0a089a44680bdfed7e

[claudepache]

Thanks. That provides arguments for including optional assignment: #18

[littledan]

This analysis is awesome!

[gisenberg]

@alangpierce Thanks so much for this detailed analysis! It's definitely appreciated.

[alangpierce]

@gisenberg asked if I could get numbers on how often out-of-scope globals are accessed in soak operations. (In CoffeeScript, a?.b evaluates to undefined rather than crashing if a is an undeclared variable and isn't a global.) They're certainly rare, but It turns out they're used more than I expected:

Total accesses of undeclared globals in soak operations: 74

Here's a gist with all 74 examples, all of which look intentional to me:
https://gist.github.com/alangpierce/9ca4eda80b148d7aba8e4b17d412f8f2

See also tc39/proposal-nullish-coalescing#13 for some more discussion about that behavior.

Note that the total amount of CoffeeScript in those repos has gone down by about 5% since my original post. I updated the README in the repo with new numbers, but the scale is pretty much the same for all of them.