Skip to content

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Oct 9, 2025

Currently we load each discovered test file to scan it for directives once for EarlyProps parsing, then reload and scan it once per revision for ignore processing. If a revision is not ignored, we then reload and scan it again during actual execution.

That's a bit silly, so this PR tries to reduce the number of unnecessary file loads and line scans for directive parsing, by reusing the same collection of DirectiveLine values for EarlyProps and for each revision's ignores.

Each individual directive still needs to be re-parsed a bunch of times, but those steps can at least avoid scanning the whole file, or having to split out names from values.


There's more that could be done after this, such as only doing known-directive checks once per file, or embedding file paths in each DirectiveLine, but I decided to stop here to allow review in modest chunks.

r? jieyouxu

@rustbot
Copy link
Collaborator

rustbot commented Oct 9, 2025

Some changes occurred in src/tools/compiletest

cc @jieyouxu

@rustbot rustbot added A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 9, 2025
@rustbot
Copy link
Collaborator

rustbot commented Oct 9, 2025

jieyouxu is currently at their maximum review capacity.
They may take a while to respond.

@jieyouxu
Copy link
Member

jieyouxu commented Oct 9, 2025

Ooo nice, I'll take a look ~tmrw.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, to me this makes much more sense, nice!

View changes since this review

@jieyouxu
Copy link
Member

Somehow I managed to click "comment" and not the "approve" button...

Anyway,
@bors r+ rollup

@bors
Copy link
Collaborator

bors commented Oct 10, 2025

📌 Commit f3f8385 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 10, 2025
bors added a commit that referenced this pull request Oct 10, 2025
Rollup of 12 pull requests

Successful merges:

 - #145651 (Regression test for const promotion with Option<Ordering>)
 - #145722 (implement Extend<{Group, Literal, Punct, Ident}> for TokenStream)
 - #146520 (Promote armv8r-none-eabihf target to Tier 2)
 - #146522 (Promote armv7a-none-eabihf to Tier 2)
 - #147289 (Mitigate `thread_local!` shadowing issues)
 - #147515 (Update rustc-perf submodule)
 - #147522 (compiletest: Use the same directive lines for EarlyProps and ignore/only/needs)
 - #147525 (Replace locals in debuginfo records during ref_prop and dest_prop)
 - #147544 (Remove StatementKind::Deinit.)
 - #147551 (remove `#[rustc_inherit_overflow_checks]` from `is_multiple_of`)
 - #147553 (Move `wasm32-wasip3`  to the tier 3 table)
 - #147562 (Stabilize `NonZero<u*>::div_ceil`)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit that referenced this pull request Oct 11, 2025
Rollup of 12 pull requests

Successful merges:

 - #145651 (Regression test for const promotion with Option<Ordering>)
 - #145722 (implement Extend<{Group, Literal, Punct, Ident}> for TokenStream)
 - #146520 (Promote armv8r-none-eabihf target to Tier 2)
 - #146522 (Promote armv7a-none-eabihf to Tier 2)
 - #147289 (Mitigate `thread_local!` shadowing issues)
 - #147515 (Update rustc-perf submodule)
 - #147522 (compiletest: Use the same directive lines for EarlyProps and ignore/only/needs)
 - #147525 (Replace locals in debuginfo records during ref_prop and dest_prop)
 - #147544 (Remove StatementKind::Deinit.)
 - #147551 (remove `#[rustc_inherit_overflow_checks]` from `is_multiple_of`)
 - #147553 (Move `wasm32-wasip3`  to the tier 3 table)
 - #147562 (Stabilize `NonZero<u*>::div_ceil`)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit cfc27fb into rust-lang:master Oct 11, 2025
10 checks passed
@rustbot rustbot added this to the 1.92.0 milestone Oct 11, 2025
rust-timer added a commit that referenced this pull request Oct 11, 2025
Rollup merge of #147522 - Zalathar:directive, r=jieyouxu

compiletest: Use the same directive lines for EarlyProps and ignore/only/needs

Currently we load each discovered test file to scan it for directives once for EarlyProps parsing, then reload and scan it once *per revision* for ignore processing. If a revision is not ignored, we then reload and scan it again during actual execution.

That's a bit silly, so this PR tries to reduce the number of unnecessary file loads and line scans for directive parsing, by reusing the same collection of `DirectiveLine` values for EarlyProps and for each revision's ignores.

Each individual directive still needs to be re-parsed a bunch of times, but those steps can at least avoid scanning the whole file, or having to split out names from values.

---

There's more that could be done after this, such as only doing known-directive checks once per file, or embedding file paths in each `DirectiveLine`, but I decided to stop here to allow review in modest chunks.

r? jieyouxu
@Zalathar Zalathar deleted the directive branch October 11, 2025 23:14
rust-cloud-vms bot pushed a commit to makai410/rustc_public that referenced this pull request Oct 12, 2025
Rollup of 12 pull requests

Successful merges:

 - rust-lang/rust#145651 (Regression test for const promotion with Option<Ordering>)
 - rust-lang/rust#145722 (implement Extend<{Group, Literal, Punct, Ident}> for TokenStream)
 - rust-lang/rust#146520 (Promote armv8r-none-eabihf target to Tier 2)
 - rust-lang/rust#146522 (Promote armv7a-none-eabihf to Tier 2)
 - rust-lang/rust#147289 (Mitigate `thread_local!` shadowing issues)
 - rust-lang/rust#147515 (Update rustc-perf submodule)
 - rust-lang/rust#147522 (compiletest: Use the same directive lines for EarlyProps and ignore/only/needs)
 - rust-lang/rust#147525 (Replace locals in debuginfo records during ref_prop and dest_prop)
 - rust-lang/rust#147544 (Remove StatementKind::Deinit.)
 - rust-lang/rust#147551 (remove `#[rustc_inherit_overflow_checks]` from `is_multiple_of`)
 - rust-lang/rust#147553 (Move `wasm32-wasip3`  to the tier 3 table)
 - rust-lang/rust#147562 (Stabilize `NonZero<u*>::div_ceil`)

r? `@ghost`
`@rustbot` modify labels: rollup
flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request Oct 18, 2025
Rollup of 12 pull requests

Successful merges:

 - rust-lang/rust#145651 (Regression test for const promotion with Option<Ordering>)
 - rust-lang/rust#145722 (implement Extend<{Group, Literal, Punct, Ident}> for TokenStream)
 - rust-lang/rust#146520 (Promote armv8r-none-eabihf target to Tier 2)
 - rust-lang/rust#146522 (Promote armv7a-none-eabihf to Tier 2)
 - rust-lang/rust#147289 (Mitigate `thread_local!` shadowing issues)
 - rust-lang/rust#147515 (Update rustc-perf submodule)
 - rust-lang/rust#147522 (compiletest: Use the same directive lines for EarlyProps and ignore/only/needs)
 - rust-lang/rust#147525 (Replace locals in debuginfo records during ref_prop and dest_prop)
 - rust-lang/rust#147544 (Remove StatementKind::Deinit.)
 - rust-lang/rust#147551 (remove `#[rustc_inherit_overflow_checks]` from `is_multiple_of`)
 - rust-lang/rust#147553 (Move `wasm32-wasip3`  to the tier 3 table)
 - rust-lang/rust#147562 (Stabilize `NonZero<u*>::div_ceil`)

r? `@ghost`
`@rustbot` modify labels: rollup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-compiletest Area: The compiletest test runner A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants