Document CI jobs

[gibfahn]

• Subsystem: doc, build

See nodejs/help#548, we should briefly document the jobs we have in CI, probably in the COLLABORATOR_GUIDE.

https://ci.nodejs.org/job/node-stress-single-test/

cc/ @vsemozhetbyt

[vsemozhetbyt]

@gibfahn I am not sure for now how to do this, need some experiments (the first attempt has failed, maybe I've configured something wrong). Maybe it would be better if anybody with more experience here documents this.

[joyeecheung]

Er, I am not sure if this is actually a good first contribution? (unless mentors are available)

[gibfahn]

@joyeecheung mentors are available 😄

I think what we need is a list of the ci jobs that collaborators (or any other contributor) need to know about. ci.nodejs.org is open to anyone to look at.

The jobs I can think of are:

• https://ci.nodejs.org/job/node-test-commit/ or https://ci.nodejs.org/job/node-test-pull-request/
  • probably already documented
• https://ci.nodejs.org/job/node-stress-single-test/
• https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/
  • possibly also documented

For each one we'd need a link and a one sentence description of what they are for. Just so that new collaborators get told this as part of Onboarding, instead of having to find out by chance later.

[vsemozhetbyt]

@gibfahn It seems the https://ci.nodejs.org/job/node-test-linter/ is also worth documenting for PRs concerning files that are not relevant for builds (like benchmarks).

[morrme]

I'm interested in helping with this, with support from a mentor.

[gibfahn]

@morrme Thanks!

So this info should be added to the CI Testing section of either CONTRIBUTING.md or the onboarding.md doc, depending on whether you think it'd be useful for people who aren't collaborators (for example, if you think this info would be useful for you as you read CONTRIBUTING.md?)

We probably want a sentence saying that our test jobs run from ci.nodejs.org, and then give links to some of the most commonly used jobs (i.e. those mentioned above).

A guide to the mechanics of raising a PR can be found in CONTRIBUTING.md as well.

If you need any help, just comment in here. A PR can also be raised for feedback before it's finished if you'd like.

[morrme]

@gibfahn Thanks! It is definitely important information for a collaborator and I personally find it interesting as a contributor as well.

So far, there are 5 tests listed.

I found a little information about node-stress_single-test from issue #3854 . Are there any other notes I can use to come up with the descriptions?

[gibfahn]

@morrme I think all you'd really need to say is that the job allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well (cc/ @Trott, we were discussing this in a PR somewhere, can't remember where).

[Trott]

@morrme I think all you'd really need to say is that the job allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well (cc/ @Trott, we were discussing this in a PR somewhere, can't remember where).

For an initial pass, the repeat info can be left off. We can add it later (or not). The main thing about node-stress-single-test is that it's designed to allow one to run a single test over and over on a specific platform to confirm that the test is reliable.

[morrme]

@gibfahn OK so for the ones you've marked as "possibly already documented", I cannot seem to find anything on these.

And for @vsemozhetbyt 's suggestion of https://ci.nodejs.org/job/node-test-linter/ : what would be a good description?

[gibfahn]

OK so for the ones you've marked as "possibly already documented", I cannot seem to find anything on these.

Okay, then please do document them as well.

https://ci.nodejs.org/job/node-test-linter/ : what would be a good description?

Maybe something like:

The node-test-linter only runs the linter targets, which is useful for changes that only affect comments or documentation.

[morrme]

OK ongoing notes:

node-stress_single-test

allows you to specify a set of tests to pass to tools/test.py, and that there's a repeat option to run a test or tests in a loop. A couple of examples might be helpful as well
it's designed to allow one to run a single test over and over on a specific platform to confirm that the test is reliable.

node-test-linter

The node-test-linter only runs the linter targets, which is useful for changes that only affect comments or documentation.

---

For node-test-pull-request
I have found #2263
Is this info still valid/current? anything to add?

There is also information there about node-test-commit but I'm still hoping to find a better description.

For citgm-smoker

Possibly use some of the NPM description: https://www.npmjs.com/package/citgm

citgm is a simple tool for pulling down an arbitrary module from npm and testing it using a specific version of the node runtime.

The Node.js project uses citgm to smoketest our releases and controversial changes.