Create proposal for list of officially supported platforms

[rvagg]

This is an extension of nodejs/node#8265 (making a new issue because we don't seem to have a build-agenda label we're using to make agenda items outside of our repo). We're being delegated the task of coming up with a proposal for an initial supported platforms / permutations list for the CTC to accept.

Would someone like to put up a hand to start such a list? We did have one on our very first README.md iteration but it's been removed for lack of maintenance (and it was aspirational rather than reflecting anything we had).

My proposal for process is (copied from nodejs/node#8265 (comment)):

Proposal from the Build WG meeting this week goes something like this:

• It's entirely up to the CTC to sign off on a "supported platforms" list
• The Build WG can be used as an expert group to (a) come up with an initial proposed list and (b) occasionally advise on updates to the list or be consulted on proposed updates. If we had an active Hardware WG then they could also be in the conversation but we don't have that right now.
• It would be good to have at least 2 tiers of hosts, perhaps even 3 (names could be improved here):
  • Officially supported: test failures on these permutations should prevent a release; master and release branches should always be green on these
  • Aspirationally supported: aiming for always-green but releases shouldn't be held up and collaborators should attempt to get to green on master and release branches but excessive yak shaving to get support should not be a holdup (e.g. getting musl features working for alpine builds?)
  • Experimental: mostly would be the Build WG playing with new platforms, no requirement for collaborators to even pay attention to these if they don't care

The trick here is the technical pieces to have different levels of support. Ideally we'd be able to prevent failures on the non-official platforms from causing a red/failed build. Perhaps we can reuse the yellow/flaky stuff somehow. The Build WG has some ideas here and some TODOs for experimentation on how to make this work. The other path is to simply deal with this complexity through the new github bot that's going to be reporting all sorts of stuff to github and may even mean collaborators don't need to even touch or look at Jenkins.

[rvagg]

Initial list might be worth taking from libuv: libuv/libuv@be0e24c (/h1 @cjihrig).

Also note FTR that this needs to go back on ctc-agenda when we have something, the CTC is waiting on us here.

[claudiorodriguez]

Not part of the workgroup but I thought it might be useful to share that Mozilla also similarly does Tier 1-3: https://developer.mozilla.org/en-US/docs/Supported_build_configurations and has some nice explanations to go with them.

[rvagg]

Nice, I particularly like the way they assign individuals responsible for tier-3 platforms, we can be doing that with the IBM platforms for instance and if anyone shows up super keen to maintain something obscure then we can give them a 👍 as long as their name gets to go next to it and they remain responsive enough to fix stuff.

[geek]

@rvagg looks good.

The list should include SmartOS as an officially supported tier 1 platform with @misterdjules and myself supporting it.

[misterdjules]

I'm not part of the working group, but I would like to add that the list of platforms supported by libuv is not necessarily a good starting point for the Node.js project with regards to the Solaris and SmartOS platforms.

The libuv project currently documents only Solaris (and not SmartOS) as a supported platform. The reason for not using the name SmartOS and for not making it a tier 1 platform, even if tests are run by the CI infrastructure on SmartOS and they all pass, might be that there's no official collaborator to the libuv project who committed to support that platform full time, even though I've been responding quickly to any issue that arose in the past.

For the Node.js project, node "sunos" binaries are actually built on SmartOS, and as a result don't run on Solaris. So they are effectively supported only on SmartOS and should really be "smartos" binaries. Finally, there are at least two official collaborators in the Node.js project who can support the SmartOS platform full-time (@geek and me).

As a result I would expect the SmartOS platform (named "SmartOS", not "SunOS", "Solaris" or with any other name) to be present in the list at tier 1 level.

[bnoordhuis]

The list should include SmartOS as an officially supported tier 1 platform with @misterdjules and myself supporting it.

I think that is stretching it a bit. IBM has a magnitude more people working on node.js and I don't think we claim that AIX/zOS/PPC are tier 1.

And no offense to you personally but I don't see many code fixes or interaction on the bug tracker vis-a-vis smartos-related issues from you. To claim tier 1 status for a platform, its supporters should be both active and proactive; I think it's fair to say that neither you nor @misterdjules really fit that description.

(I'm not attacking or berating you, just stating my perceptions. If you think I'm wrong, please say so.)

[mhdawson]

Just to throw in my 2 cents in, I like the Mozilla 1-3 but I would like to see the IBM platforms getting to Tier 2 as I think we do better than Tier 3 implies. There is hardware in the CI for these platforms and pretty much all tests are running on them as well.

[misterdjules]

The list should include SmartOS as an officially supported tier 1 platform with @misterdjules and myself supporting it.

I think that is stretching it a bit. IBM has a magnitude more people working on node.js and I don't think we claim that AIX/zOS/PPC are tier 1.

I don't think comparing the number of people employed by some of the companies that drive the development of some platforms on which node runs is a good way to approach this problem.

It seems that the reason why some of the maintainers of the AIX/zOS/PPC platforms may not be considering these platforms as part of tier 1 yet is that they've been added relatively recently to the CI platform compared to other platforms such as SmartOS, and they still have more tests failing and/or flaky than most other platforms.

In comparison, tests for the SmartOS platform have been running for years and now run with a number of flaky tests that is comparable to other platforms, thanks among other things to the great work done by @Trott, @jbergstroem, @santigimeno, the @nodejs/platform-solaris team and others.

And no offense to you personally but I don't see many code fixes or interaction on the bug tracker vis-a-vis smartos-related issues from you.

@geek joined Joyent only two weeks ago, and thus just started to have the time resources to focus on SmartOS since then, so it's not a surprise you haven't seen him active on that platform yet. But Joyent hiring him is a clear sign of its commitment to maintaining Node.js on SmartOS.

To claim tier 1 status for a platform, its supporters should be both active and proactive; I think it's fair to say that neither you nor @misterdjules really fit that description.

I've been active and proactive in supporting the SmartOS platform, in nodejs/node, libuv/libuv but also in other projects like nvm. When not directly involved in fixing issues that are specific to SmartOS, I've provided support and resources to the build team for issues related to the SmartOS platform on a regular basis. Several other collaborators that I mentioned above have done the same.

This is why the level of support for the SmartOS platform in the Node.js project matches both libuv's and Mozilla's definition of a Tier 1 platform.

If that level of support is not considered enough, then we should work together to bring it to a level that satisfies the expectations. An example of such an effort is libuv/libuv#1036, where I'm trying to improve the communication channels between the libuv collaborators team and maintainers of the SmartOS platform.

[mhdawson]

I think that only AIX has more flaky tests (which we are working on). PPC and linuxOne run consistently clean.

[jbergstroem]

Here's a initial stab at a "supported platform" document: https://gist.github.com/jbergstroem/8a4a5b6a602aacc96eb8a171698c324c

[Trott]

• What's the justification for SmartOS being Experimental rather than Tier 2? (I see lots of discussion about whether or not its Tier 1 above, but if it's not even Tier 2, a clear statement of the gap would be helpful. I'm not invested in SmartOS or anything, and I don't really much care where it ends up, but I am surprised to see it as Experimental given the descriptions of the different tiers in the doc. Perhaps I am misunderstanding something somewhere?)
• Perhaps related: Does "There needs to be at least one individual..." refer to an individual in the project or an individual in the Build WG?

[jbergstroem]

@Trott said:
What's the justification for SmartOS being Experimental rather than Tier 2?

No one from the build group has compiled or tested Node.js on Smartos 15 and 16.