Add new --bypass-ignore-backends option

[GuillaumeGomez]

Fixes #147063.

It adds a new option to bootstrap to allow to ignore //@ ignore-backends statements in tests.

cc @jieyouxu @antoyo
r? @Kobzol

[rustbot]

Some changes occurred in src/tools/compiletest

cc @jieyouxu

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

[Kobzol]

To provide more context here, I suggested this to allow gradual removal of ignore-backends: gcc in a more widespread manner. The idea is that the GCC backend will evolve over time, and hopefully it will be able to run more and more tests. It would be great if we had an ability to run tests that were failing before, and if they succeed now, remove ignore-backends: gcc from them.

That being said, now that I think about it, I wonder how would the workflow look like. If you use the option that this PR adds, it would just run all tests; some of them would fail, some of them would succeed. But it wouldn't be trivial to figure out which one of those that succeeded actually contain ignore-backends: gcc; I expect that GCC will usually resolve a bunch of previous test failures at once, not just individual tests, so going through them manually might be cumbersome, and at that point you could just comment out or remove ignore-backends: gcc yourself.

I guess what I really wanted was something like treat ignore-backends: gcc as "failure is expected" and do not apply the ignore mechanism. Then we would run these tests with the GCC backend, and those tests that would fail are essentially newly passing tests that should have their ignore-backends: gcc removed. Seems like this might be useful as a more general utility in compiletest, rather than hardcoding it for ignore-backends specifically?

@jieyouxu Let me know if you have any opinions on this.

[antoyo]

Here are the 3 use cases we currently have in the CI of cg_gcc I can think of:

• Enable the LTO tests for the CI tasks where the sysroot is compiled with LTO.
• Only run the ignored tests to know which tests now pass.
• Run some specific ignored tests because they now pass.

We already have the logic to do this in the CI of cg_gcc, but we need a flag like --bypass-ignore-backends.

[Kobzol]

Maybe we could do a general override mechanism, something like --bypass-ignore=FOO[=BAR], which would ignore the ignore directives foo: bar.

[antoyo]

Maybe we could do a general override mechanism, something like --bypass-ignore=FOO[=BAR], which would ignore the ignore directives foo: bar.

I guess that makes sense if there are other ignore directives that we would want to do that. I know some ignore directives are related to the architecture, so it doesn't make much sense, but maybe there are others where it would make sense?

[jieyouxu]

Maybe we could do a general override mechanism, something like --bypass-ignore=FOO[=BAR], which would ignore the ignore directives foo: bar.

I'm hesitant about generalizing this to be a general ignore-override mechanism, for a few reasons:

• ignore-* and needs-* tend to come in pairs.
• ignore-*s can be due to several causes, for instance:
  • "This test fails only on $platform and I don't know why"
  • "This test can never sensibly work on $platform because we're testing $platform-specific things
  • "This test only makes sense and is intended only for the non-ignored set of conditions"

"This test fails only on $ignored_conditions and I don't know why" is the one such an "ignore-override" would generally be designed for, but it feels a bit too generic. Another use case might be "this conditional directive is unneeded", but that's hard to assess because it might not fail in CI, but it might fail locally for non-CI-tested configurations without said directive.

[jieyouxu]

Suggestion: if we do go with this, can you please document this alongside //@ ignore-backends in rustc-dev-guide?

[jieyouxu]

Nit: s/statements/directives/.

[jieyouxu]

Nit: probably "bypass", ignoring ignore sounds a bit confusing at first glance.

[jieyouxu]

Given that

We already have the logic to do this in the CI of cg_gcc

This seems okay, though please do document this flag and its intended purpose in rustc-dev-guide, or else no one else will have clues on what it's supposed to do.