Codegen of experiments.g.dart doesn't appear to be unit tested

[stereotype441]

In https://dart-review.googlesource.com/c/sdk/+/182300 I fixed a minor null safety bug in the code generation logic in pkg/analyzer/tool/experiments/generate.dart (which generates pkg/analyzer/lib/src/dart/analysis/experiments.g.dart). The fix prevented a crash when running the code generator. Normally we have a unit test to verify that the output of a code generator matches the corresponding generated file, so that we don't forget to make fixes like this one. It looks like we don't have a unit test for experiments.g.dart.

[scheglov]

I look at bots/test_matrix.json, there is

       {
          "name": "analyzer unit tests",
          "arguments": [
            "-nanalyzer-unittest-asserts-${mode}-${system}",
            "pkg/analyzer"
          ],
          "shards": 2,
          "fileset": "analyzer_unit_tests"
        },

When I run tools/test.py -nanalyzer-unittest-asserts-release-mac pkg/analyzer locally, and my experiments.g.dart is changed, I get the expected failure.

FAILED: dartk-vm release_x64 pkg/analyzer/tool/experiments/experiments_test
Expected: Pass
Actual: Fail

--- Command "vm" (took 02.000806s):
DART_CONFIGURATION=ReleaseX64 xcodebuild/ReleaseX64/dart-sdk/bin/dart --enable_asserts --ignore-unrecognized-flags --packages=/Users/scheglov/Source/Dart/sdk.git/sdk/.packages /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/tool/experiments/experiments_test.dart

exit code:
1

stdout:
File: '/Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/lib/src/dart/analysis/experiments.g.dart' doesn't have expected contents.
Please regenerate using:
  xcodebuild/ReleaseX64/dart-sdk/bin/dart /Users/scheglov/Source/Dart/sdk.git/sdk/pkg/analyzer/tool/experiments/generate.dart

--- Re-run this test:
python tools/test.py -n analyzer-unittest-asserts-release-mac pkg/analyzer/tool/experiments/experiments_test

I will try to upload a CL with changed experiments.g.dart and see if this makes bots red.

[scheglov]

The reason it was not failing is that we did not run the test in null safe mode.
FWIW https://dart-review.googlesource.com/c/sdk/+/183780 goes purple.

https://dart-review.googlesource.com/c/sdk/+/183764 should fix it.

[stereotype441]

The reason it was not failing is that we did not run the test in null safe mode.
FWIW https://dart-review.googlesource.com/c/sdk/+/183780 goes purple.

https://dart-review.googlesource.com/c/sdk/+/183764 should fix it.

Ah, that makes sense! Thanks for tracking this down!