Add `#[loop_match]` for improved DFA codegen #138780

folkertdev · 2025-03-21T11:40:03Z

tracking issue: #132306
project goal: rust-lang/rust-project-goals#258

This PR adds the #[loop_match] attribute, which aims to improve code generation for state machines. For some (very exciting) benchmarks, see rust-lang/rust-project-goals#258 (comment)

Currently, a very restricted syntax pattern is accepted. We'd like to get feedback and merge this now before we go too far in a direction that others have concerns with.

current state

We accept code that looks like this

#[loop_match]
loop {
    state = 'blk: {
        match state {
            State::A => {
                #[const_continue]
                break 'blk State::B
            }
            State::B => { /* ... */ }
            /* ... */
        }
    }
}

a loop should have the same semantics with and without #[loop_match]: normal continue and break continue to work
#[const_continue] is only allowed in loops annotated with #[loop_match]
the loop body needs to have this particular shape (a single assignment to the match scrutinee, with the body a labelled block containing just a match)

future work

perform const evaluation on the break value
support more state/scrutinee types

maybe future work

allow continue 'label value syntax, which #[const_continue] could then use.
allow the match to be on an arbitrary expression (e.g. State::Initial)
attempt to also optimize break/continue expressions that are not marked with #[const_continue]

r? @traviscross

rustbot · 2025-03-21T11:40:10Z

Some changes occurred in match checking

cc @Nadrieril

Some changes occurred in compiler/rustc_passes/src/check_attr.rs

cc @jdonszelmann

Some changes occurred in rustc_ty_utils::consts.rs

cc @BoxyUwU

traviscross

Thanks @folkertdev for putting up this PR. The big picture looks right, in terms of the behavior of the tests and how to approach the experiment in terms of starting with the attributes for thiis.

This is a first partial pass on the details.

@rustbot author

compiler/rustc_feature/src/unstable.rs

compiler/rustc_mir_build/messages.ftl

tests/ui/loop-match/break-to-block.rs

compiler/rustc_mir_build/src/builder/expr/into.rs

compiler/rustc_mir_build/src/builder/scope.rs

compiler/rustc_passes/messages.ftl

compiler/rustc_middle/src/thir.rs

folkertdev

Thanks for the detailed review!

I've fixed a bunch of the low-hanging fruit (e.g. in the tests). For the actual pattern matching logic, I have a branch with what I believe is a better solution that re-uses more existing pattern matching infra. We'll come back to that here once björn has had a chance to look at it.

compiler/rustc_mir_build/src/builder/expr/into.rs

compiler/rustc_middle/src/thir.rs

rustbot · 2025-03-24T10:46:56Z

Some changes occurred in exhaustiveness checking

cc @Nadrieril

Some changes occurred in match lowering

cc @Nadrieril

compiler/rustc_mir_build/src/builder/expr/stmt.rs

bors · 2025-03-26T15:19:18Z

☔ The latest upstream changes (presumably #138974) made this pull request unmergeable. Please resolve the merge conflicts.

compiler/rustc_mir_build/src/builder/scope.rs

folkertdev · 2025-04-04T13:12:50Z

We've done a bunch of work here, and I believe all of the earlier review comments have now been dealt with.

@rustbot ready

compiler/rustc_feature/src/builtin_attrs.rs

compiler/rustc_feature/src/unstable.rs

compiler/rustc_middle/src/thir.rs

compiler/rustc_middle/src/thir/visit.rs

compiler/rustc_mir_build/src/builder/matches/mod.rs

compiler/rustc_mir_build/src/builder/expr/into.rs

compiler/rustc_mir_build/src/builder/matches/mod.rs

compiler/rustc_mir_build/src/builder/scope.rs

traviscross · 2025-04-06T03:01:58Z

@rustbot author

As a lang matter, this is looking reasonable to me in terms of a lang experiment.

As an impl matter, this is starting to look not unreasonable to me, but I'd like for @Nadrieril to also have a look if he's able.

r? @Nadrieril

@Nadrieril: I still need to raise this in a lang meeting to confirm that everyone is happy to see the experiment here in light of earlier objections, so please don't merge this just yet. You can leave it back in my hands after you're happy with the impl.

Also CC @oli-obk as this work is carrying over some FIXME items you have marked.

rustbot · 2025-04-06T03:02:02Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

bors · 2025-06-23T16:28:49Z

☔ The latest upstream changes (presumably #142906) made this pull request unmergeable. Please resolve the merge conflicts.

Co-authored-by: Folkert de Vries <[email protected]>

lcnr · 2025-06-24T07:43:07Z

don't have too much time rn and properly reviewing this PR is effort

r? compiler

If this doesn't get reviewed over the next ~2 weeks reassign to me

oli-obk

Seems fine to me as an experiment, but I don't see how to turn it into an actual feature later without outright replacing most of it and its tests. That doesn't mean this is badly coded or anything, just that adding first class syntax will make a rewrite very desirable to carry the higher level information properly downwards and avoid lots of situations that have to recover that information in lossy ways

oli-obk · 2025-06-24T13:35:00Z

compiler/rustc_mir_build/src/builder/scope.rs

@@ -742,6 +816,190 @@ impl<'a, 'tcx> Builder<'a, 'tcx> {
        self.cfg.start_new_block().unit()
    }

+    /// Based on `FunctionCx::eval_unevaluated_mir_constant_to_valtree`.


why is this duplicated and not shared?

We didn't see a good way to share the code. The original is a private method on FunctionCx in rustc_codegen_ssa, we're in rustc_mir_build, so that function does some things (like .monomorphize) that we can't yet do here.

oli-obk · 2025-06-24T13:47:20Z

compiler/rustc_mir_build/src/builder/scope.rs

+        };
+
+        let Some(real_target) =
+            self.static_pattern_match(&cx, valtree, &*scope.arms, &scope.built_match_tree)


yea I don't really have an idea for how to make this work well in general without redesigning the feature from scratch independently from pattern matching. Not sure how and what parts of the CTFE machinery we could extract out to have something reusable as it works on MIR and thinks only switch statements on integers exist.

We only need to evaluate the scrutinee into some sort of value representation. I guess I don't really understand what you think the problem is here?

my "complaint" is that this is new logic not shared with anything else in the compiler and that we likely won't share with anything else, even tho we do similar things e.g. in const eval. But there we do it post-mir-building and "just" process the built MIR. I just worry we'll have weird bugs if we accidentally diverge in some less tested use cases

compiler/rustc_mir_build/src/builder/matches/mod.rs

oli-obk · 2025-06-24T13:53:46Z

compiler/rustc_mir_build/src/builder/matches/mod.rs

+            let pat = cx.lower_pat(&*self.thir.arms[arm_id].pattern);
+
+            // Peel off or-patterns if they exist.
+            if let rustc_pattern_analysis::rustc::Constructor::Or = pat.ctor() {


couldn't this be part of static_pattern_match_inner and just recurse on itself in Or patterns?

This peels off only the outer layer. Nested or-patterns are currently impossible to handle, because we can't always associate the pattern with the correct branch. @Nadrieril had some mid/long-term ideas of how to refactor the or-pattern logic so that this connection can be made.

Given the types that we accept now, only handling top-level or-patterns isn't a limitation, and so we're again defensive here and only implement that which we can guarantee will work.

folkertdev

Thanks for the review!

You're right that we'd need to make a bunch of changes if syntax were added. However, discussions about syntax are just endless (everyone can have an opinion about syntax, way fewer people weigh in on the technical details). So syntax is not something that I think we'll push for for a while: the technical side has to be promising before we'll bother with that.

Having loop_match on nightly will make it a lot easier to experiment with.

folkertdev · 2025-06-24T14:16:37Z

compiler/rustc_mir_build/src/builder/scope.rs

@@ -742,6 +816,190 @@ impl<'a, 'tcx> Builder<'a, 'tcx> {
        self.cfg.start_new_block().unit()
    }

+    /// Based on `FunctionCx::eval_unevaluated_mir_constant_to_valtree`.


We didn't see a good way to share the code. The original is a private method on FunctionCx in rustc_codegen_ssa, we're in rustc_mir_build, so that function does some things (like .monomorphize) that we can't yet do here.

folkertdev · 2025-06-24T14:20:10Z

compiler/rustc_mir_build/src/builder/scope.rs

+        };
+
+        let Some(real_target) =
+            self.static_pattern_match(&cx, valtree, &*scope.arms, &scope.built_match_tree)


We only need to evaluate the scrutinee into some sort of value representation. I guess I don't really understand what you think the problem is here?

compiler/rustc_mir_build/src/builder/matches/mod.rs

folkertdev · 2025-06-24T14:31:40Z

compiler/rustc_mir_build/src/builder/matches/mod.rs

+            let pat = cx.lower_pat(&*self.thir.arms[arm_id].pattern);
+
+            // Peel off or-patterns if they exist.
+            if let rustc_pattern_analysis::rustc::Constructor::Or = pat.ctor() {


This peels off only the outer layer. Nested or-patterns are currently impossible to handle, because we can't always associate the pattern with the correct branch. @Nadrieril had some mid/long-term ideas of how to refactor the or-pattern logic so that this connection can be made.

Given the types that we accept now, only handling top-level or-patterns isn't a limitation, and so we're again defensive here and only implement that which we can guarantee will work.

oli-obk · 2025-06-24T14:50:43Z

@bors r+

bors · 2025-06-24T14:50:47Z

📌 Commit ba5556d has been approved by oli-obk

It is now in the queue for this repository.

…attr, r=oli-obk Add `#[loop_match]` for improved DFA codegen tracking issue: rust-lang#132306 project goal: rust-lang/rust-project-goals#258 This PR adds the `#[loop_match]` attribute, which aims to improve code generation for state machines. For some (very exciting) benchmarks, see rust-lang/rust-project-goals#258 (comment) Currently, a very restricted syntax pattern is accepted. We'd like to get feedback and merge this now before we go too far in a direction that others have concerns with. ## current state We accept code that looks like this ```rust #[loop_match] loop { state = 'blk: { match state { State::A => { #[const_continue] break 'blk State::B } State::B => { /* ... */ } /* ... */ } } } ``` - a loop should have the same semantics with and without `#[loop_match]`: normal `continue` and `break` continue to work - `#[const_continue]` is only allowed in loops annotated with `#[loop_match]` - the loop body needs to have this particular shape (a single assignment to the match scrutinee, with the body a labelled block containing just a match) ## future work - perform const evaluation on the `break` value - support more state/scrutinee types ## maybe future work - allow `continue 'label value` syntax, which `#[const_continue]` could then use. - allow the match to be on an arbitrary expression (e.g. `State::Initial`) - attempt to also optimize `break`/`continue` expressions that are not marked with `#[const_continue]` r? `@traviscross`

traviscross · 2025-06-24T23:47:55Z

@bors r-
@bors r=oli-obk,traviscross

bors · 2025-06-24T23:47:59Z

📌 Commit ba5556d has been approved by oli-obk,traviscross

It is now in the queue for this repository.

…attr, r=oli-obk,traviscross Add `#[loop_match]` for improved DFA codegen tracking issue: rust-lang#132306 project goal: rust-lang/rust-project-goals#258 This PR adds the `#[loop_match]` attribute, which aims to improve code generation for state machines. For some (very exciting) benchmarks, see rust-lang/rust-project-goals#258 (comment) Currently, a very restricted syntax pattern is accepted. We'd like to get feedback and merge this now before we go too far in a direction that others have concerns with. ## current state We accept code that looks like this ```rust #[loop_match] loop { state = 'blk: { match state { State::A => { #[const_continue] break 'blk State::B } State::B => { /* ... */ } /* ... */ } } } ``` - a loop should have the same semantics with and without `#[loop_match]`: normal `continue` and `break` continue to work - `#[const_continue]` is only allowed in loops annotated with `#[loop_match]` - the loop body needs to have this particular shape (a single assignment to the match scrutinee, with the body a labelled block containing just a match) ## future work - perform const evaluation on the `break` value - support more state/scrutinee types ## maybe future work - allow `continue 'label value` syntax, which `#[const_continue]` could then use. - allow the match to be on an arbitrary expression (e.g. `State::Initial`) - attempt to also optimize `break`/`continue` expressions that are not marked with `#[const_continue]` r? `@traviscross`

Rollup of 15 pull requests Successful merges: - #135731 (Implement parsing of pinned borrows) - #138780 (Add `#[loop_match]` for improved DFA codegen) - #142453 (Windows: make `read_dir` stop iterating after the first error is encountered) - #142633 (Error on invalid signatures for interrupt ABIs) - #142768 (Avoid a bitcast FFI call in transmuting) - #142825 (Port `#[track_caller]` to the new attribute system) - #142844 (Enable short-ice for Windows) - #142934 (Tweak `-Zmacro-stats` measurement.) - #142955 (Couple of test suite fixes for cg_clif) - #142977 (rustdoc: Don't mark `#[target_feature]` functions as ⚠) - #142980 (Reduce mismatched-lifetime-syntaxes suggestions to MaybeIncorrect) - #142982 (Corrected spelling mistake in c_str.rs) - #142983 (Taint body on invalid call ABI) - #142988 (Update wasm-component-ld to 0.5.14) - #142993 (Update cargo) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #138780 - trifectatechfoundation:loop_match_attr, r=oli-obk,traviscross Add `#[loop_match]` for improved DFA codegen tracking issue: #132306 project goal: rust-lang/rust-project-goals#258 This PR adds the `#[loop_match]` attribute, which aims to improve code generation for state machines. For some (very exciting) benchmarks, see rust-lang/rust-project-goals#258 (comment) Currently, a very restricted syntax pattern is accepted. We'd like to get feedback and merge this now before we go too far in a direction that others have concerns with. ## current state We accept code that looks like this ```rust #[loop_match] loop { state = 'blk: { match state { State::A => { #[const_continue] break 'blk State::B } State::B => { /* ... */ } /* ... */ } } } ``` - a loop should have the same semantics with and without `#[loop_match]`: normal `continue` and `break` continue to work - `#[const_continue]` is only allowed in loops annotated with `#[loop_match]` - the loop body needs to have this particular shape (a single assignment to the match scrutinee, with the body a labelled block containing just a match) ## future work - perform const evaluation on the `break` value - support more state/scrutinee types ## maybe future work - allow `continue 'label value` syntax, which `#[const_continue]` could then use. - allow the match to be on an arbitrary expression (e.g. `State::Initial`) - attempt to also optimize `break`/`continue` expressions that are not marked with `#[const_continue]` r? ``@traviscross``

rustbot assigned traviscross Mar 21, 2025

rustbot added A-attributes Area: Attributes (`#[…]`, `#![…]`) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 21, 2025

traviscross reviewed Mar 22, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 22, 2025

This comment has been minimized.

Sign in to view

folkertdev commented Mar 22, 2025

View reviewed changes

compiler/rustc_mir_build/src/builder/expr/into.rs Outdated Show resolved Hide resolved

compiler/rustc_mir_build/src/builder/expr/into.rs Outdated Show resolved Hide resolved

compiler/rustc_middle/src/thir.rs Outdated Show resolved Hide resolved

folkertdev commented Mar 24, 2025

View reviewed changes

compiler/rustc_mir_build/src/builder/expr/stmt.rs Outdated Show resolved Hide resolved

folkertdev force-pushed the loop_match_attr branch from 368f722 to a89dcbe Compare March 27, 2025 09:39

This comment has been minimized.

Sign in to view

folkertdev commented Apr 1, 2025

View reviewed changes

compiler/rustc_mir_build/src/builder/scope.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

folkertdev force-pushed the loop_match_attr branch from f294773 to 6fe6909 Compare April 1, 2025 16:53

This comment has been minimized.

Sign in to view

folkertdev force-pushed the loop_match_attr branch from b3a87ed to 7d88da4 Compare April 4, 2025 08:22

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 4, 2025

traviscross mentioned this pull request Apr 6, 2025

Tracking issue for way to express intraprocedural finite state machines #132306

Open

7 tasks

traviscross reviewed Apr 6, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 6, 2025

rustbot assigned Nadrieril Apr 6, 2025

rustbot unassigned BoxyUwU Jun 23, 2025

Add #[loop_match] for improved DFA codegen

ba5556d

Co-authored-by: Folkert de Vries <[email protected]>

folkertdev force-pushed the loop_match_attr branch from 0025cda to ba5556d Compare June 23, 2025 18:43

rustbot assigned oli-obk and unassigned lcnr Jun 24, 2025

oli-obk approved these changes Jun 24, 2025

View reviewed changes

folkertdev commented Jun 24, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 24, 2025

workingjubilee mentioned this pull request Jun 24, 2025

Rollup of 15 pull requests #142971

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 24, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 24, 2025

This was referenced Jun 25, 2025

Rollup of 15 pull requests #142995

Closed

Rollup of 15 pull requests #142997

Merged

workingjubilee closed this Jun 25, 2025

workingjubilee reopened this Jun 25, 2025

bors merged commit f542909 into rust-lang:master Jun 25, 2025
20 checks passed

rustbot added this to the 1.90.0 milestone Jun 25, 2025

Add #[loop_match] for improved DFA codegen #138780

Add #[loop_match] for improved DFA codegen #138780

Conversation

folkertdev commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

current state

future work

maybe future work

Uh oh!

rustbot commented Mar 21, 2025

Uh oh!

traviscross left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

folkertdev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rustbot commented Mar 24, 2025

Uh oh!

Uh oh!

bors commented Mar 26, 2025

Uh oh!

This comment has been minimized.

Uh oh!

This comment has been minimized.

This comment has been minimized.

folkertdev commented Apr 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

traviscross commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Apr 6, 2025

Uh oh!

bors commented Jun 23, 2025

Uh oh!

lcnr commented Jun 24, 2025

Uh oh!

oli-obk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

folkertdev left a comment

Add `#[loop_match]` for improved DFA codegen #138780

Add `#[loop_match]` for improved DFA codegen #138780

folkertdev commented Mar 21, 2025 •

edited

Loading

traviscross commented Apr 6, 2025 •

edited

Loading