test(node): New Node integration test runner

[timfish]

To aid in adding tests for #9907

Adds a new Node integration test runner that runs the scenario in its own Node process and reads the envelope back from stdout.

I have converted all the Anr and LocalVariables tests to use this new format.

How to use

First, in your test scenario, you need to set the transport to the loggingTransport from @sentry-internal/node-integration-tests. This will cause envelopes to be logged to stdout as a single line of JSON:

import * as Sentry from '@sentry/node';
import { loggingTransport } from '@sentry-internal/node-integration-tests';

Sentry.init({
  dsn: 'https://[email protected]/1337',
  transport: loggingTransport,
});

Then in your test, use the createRunner function to create your test. The createRunner args are the path segments to the scenario. If you pass a typescript entry file, -r ts-node/register gets added to the node args.

Every expect call adds an expected envelope item. Currently you can expect event, transaction and session but it's easy to add extra type-safe expectations. You must add an expect for every envelope item that will be sent and they must be in the correct order or an error is thrown and the test fails. The expect event/transaction/session can be a callback or an object that will be tested against the received envelope.

If you don't care about specific types of envelope item, you can ignore them:

createRunner(__dirname, 'basic.js')
  .ignore('session', 'sessions')
  .expect(//...

If you pass done to start, done will be called when all the envelope items have matched the expected and done will be passed any errors that occur which ends the test without waiting for the timeout.

Example

Here is an Anr test that ensures we get the expected session envelope followed by the expected event envelope:

import { assertSentryEvent, assertSentrySession, createRunner } from '../../utils/runner';

const EXPECTED_ANR_EVENT = {
  // Ensure we have context
  contexts: {
    trace: {
      span_id: expect.any(String),
      trace_id: expect.any(String),
    },
    device: {
      arch: expect.any(String),
    },
    app: {
      app_start_time: expect.any(String),
    },
    os: {
      name: expect.any(String),
    },
    culture: {
      timezone: expect.any(String),
    },
  },
  // and an exception that is our ANR
  exception: {
    values: [
      {
        type: 'ApplicationNotResponding',
        value: 'Application Not Responding for at least 200 ms',
        mechanism: { type: 'ANR' },
        stacktrace: {
          frames: expect.arrayContaining([
            {
              colno: expect.any(Number),
              lineno: expect.any(Number),
              filename: expect.any(String),
              function: '?',
              in_app: true,
            },
            {
              colno: expect.any(Number),
              lineno: expect.any(Number),
              filename: expect.any(String),
              function: 'longWork',
              in_app: true,
            },
          ]),
        },
      },
    ],
  },
};

test('Anr with session', done => {
  createRunner(__dirname, 'basic-session.js')
    .expect({ session: { status: 'abnormal', abnormal_mechanism: 'anr_foreground' } })
    .expect({ event:  EXPECTED_ANR_EVENT })
    .start(done);
});

It's also possible to pass flags to node via the withFlags() function:

  test('With --inspect',done => {
    createRunner(__dirname, 'basic.mjs')
      .withFlags('--inspect')
      .expect({ event: EXPECTED_ANR_EVENT })
      .start(done);
  });

[mydea]

this sounds nice! Does that work with OpenTelemetry, I wonder? So with node-experimental? Would be a great way to write tests for integrations etc...!

[timfish]

Does that work with OpenTelemetry, I wonder? So with node-experimental?

Yep, that's partly what its for:

  test('otel with loader', done => {
    createRunner(__dirname, 'express-otel.mjs')
      .withFlags('--experimental-loader=@opentelemetry/instrumentation/hook.mjs')
      .expect({ transaction: EXPRESS_TRANSACTION })
      .start(done);
  });

[timfish]

I've added server request support and modified some of the express tracing tests to use the new runner.

To support server requests, make the following changes to your test scenario:

It can then be tested by calling makeRequest:

test('should create and send transactions for Express routes and spans for middlewares.', done => {
  createRunner(__dirname, 'server.ts')
    .expect({
      transaction: {
        contexts: {
          trace: {
            span_id: expect.any(String),
            trace_id: expect.any(String),
            data: {
              url: '/test/express',
              'http.response.status_code': 200,
            },
            op: 'http.server',
            status: 'ok',
            tags: {
              'http.status_code': '200',
            },
          },
        },
        spans: [
          expect.objectContaining({
            description: 'corsMiddleware',
            op: 'middleware.express.use',
          }),
        ],
      },
    })
    .start(done)
    .makeRequest('get', '/test/express');
});

[timfish]

I added this simply because it allows you to call yarn jest -t "name-of-test" --verbose and get debug output very quickly from a single test. yarn test -t "name-of-test" runs through the existing custom test runner code and takes ages...

Hopefully we can migrate to just using the jest runner!

[lforst]

Looks good!