Save x.py's help text for saving output time #146692

Shunpoco · 2025-09-17T22:20:16Z

Fix #141903

Currently x.py help (--help) builds bootstrap binary everytime, so it takes some seconds to print help.
This PR does:

Saves current help text into a file (x.py run generate-help)
Changes bootstrap.py to print the help in the saved file and to exit without touching bootstrap binary
Modifies x.py test bootstrap to check if the help file is up-to-date

rustbot · 2025-09-17T23:48:11Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

rustbot · 2025-09-17T23:48:13Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

jieyouxu · 2025-09-18T04:14:49Z

I'm on the fence on this. I'm slightly leaning towards "I don't feel like the extra complexity and having to remember another point of potential outdatedness pulls its weight". But if another bootstrap reviewer feels okay about this, that is fine with me too.

Kobzol · 2025-09-18T18:34:53Z

I also wasn't a super big fan of this, tbh. But while testing this PR, I realized that this approach actually cannot work in the way it was designated in the original issue, or at least not generally. Because it assumes that bootstrap options and flags are unified, but that is no longer the case. Since we switched to clap, some subcommands actually have their own arguments, and thus also different help.

For example, here is the output of x perf --help:

Perform profiling and benchmarking of the compiler using `rustc-perf`

Usage: x.py perf [OPTIONS] [PATHS]... [-- <ARGS>...] <COMMAND>

Commands:
  eprintln    Run `profile_local eprintln`. This executes the compiler on the given benchmarks and stores its stderr output
  samply      Run `profile_local samply` This executes the compiler on the given benchmarks and profiles it with `samply`. You need to install `samply`, e.g. using `cargo install samply`
  cachegrind  Run `profile_local cachegrind`. This executes the compiler on the given benchmarks under `Cachegrind`
  benchmark   Run compile benchmarks with a locally built compiler
  compare     Compare the results of two previously executed benchmark runs

Arguments:
  [PATHS]...
          paths for the subcommand

  [ARGS]...
          arguments passed to subcommands

Options:
  -v, --verbose...
          use verbose output (-vv for very verbose)

  -i, --incremental
          use incremental compilation

      --config <FILE>
          TOML configuration file for build

      --build-dir <DIR>
          Build directory, overrides `build.build-dir` in `bootstrap.toml`

... many shared bootstrap flags

But with this PR, when I do x perf --help, it will only print the saved help of x --help, which is wrong.

So for this to work generically, the Python wrapper would have to:

Recognize which bootstrap command is being executed (at least for the commands that have some custom arguments/options/subcommands)
Save the help output for each such command specifically

Even 1) is annoying, as the Python wrapper would have to reimplement CLI parsing logic, to recognize e.g. things like ./x --stage 1 build. But in any case, I don't think we should have that kind of complexity in the Python layer.

Sorry for not catching that earlier, but I think that this is not going to work. CC @jyn514 if you have other opinions.

Shunpoco · 2025-09-18T19:00:22Z

Thanks, I didn't catch x <subcommand> --help also goes to the saved file.

And

Recognize which bootstrap command is being executed (at least for the commands that have some custom arguments/options/subcommands)

I agree, this is inevitable if all subcommands' helps are also loaded from files. I don't think this is a good way.

I think we can make only root help (x help|--help|-h) use a saved file by changing help_triggered in bootstrap.py. In that case, only the root speeds up to print the help but other subcommands still use helps from the binary.
Of course I need to confirm if that is enough valuable to bring the complexity, though.

Mark-Simulacrum · 2025-09-21T17:47:32Z

Per above I think we can probably close this?

One thought is we could plausibly generate e.g. the man pages that clap supports and check those in, but it all seems fairly complicated for not as much value.

jyn514 · 2025-09-23T15:48:10Z

I think if we only do this for the very top-level x.py --help that's fine. We can still delegate to the rust code for subcommand help (anything other than the exact string -h or --help), that doesn't require the python to be smart.

Shunpoco · 2025-09-23T18:41:54Z

@jyn514 Thanks for the confirmation,
fb5b849 makes only root help use a saved file (help for subcommands go to the binary)

src/bootstrap/src/core/config/flags.rs

src/bootstrap/bootstrap.py

Mark-Simulacrum · 2025-09-28T14:48:01Z

src/etc/xhelp

+      --skip-std-check-if-no-download-rustc
+          Skip checking the standard library if `rust.download-rustc` isn't available. This is mostly for RA as building the stage1 compiler to check the library tree on each code change might be too much for some computers
+  -h, --help
+          Print help (see more with '--help')


Probably not relevant to this PR, but I wonder if we should really be showing all of this in help... e.g., the PGO options don't feel relevant to 99% of x.py users, and there's enough options here that we end up scrolling off screen for the actually-useful collection of subcommands test/doc/check/etc.

It might mean that we want a verbose and non-verbose help for the top level?

It makes sense to me for both > not relevant to this PR and > want a verbose and non-verbose help for the top level

Although I don't know if Clap supports such detailed/brief help, can I make an issue ticket for it (Check Clap's features, then modify the top-level help if we can)

Shunpoco · 2025-10-01T09:33:17Z

@rustbot review

Note: I will clean up the commit history after the review

Mark-Simulacrum · 2025-10-01T14:46:01Z

src/bootstrap/bootstrap.py


-    # If the user is asking for help, let them know that the whole download-and-build
+    # Root help (e.g., x.py --help) prints help from the saved file to save the time
+    if len(sys.argv) == 1 or sys.argv[1] in ["-h", "--help"]:


The second part of this seems like it's not quite right, it needs (sys.argv[1] in ["-h", "--help"] and len(sys.argv) == 2), right? Otherwise x.py --help check wouldn't do the right thing?

Currently x.py --help check prints the top level help, not check's help. So I think we don't need len(sys.argv) == 2 for it.
Or, do you think x.py --help check should be identical with x.py check --help?

Or, do you think~

I guess if we dispatch x.py --help check to bootstrap binary. Clap returns the top level help for it unless we modify the behaviour.

Or, do you think x.py --help check should be identical with x.py check --help?

Yeah, that's what I'd expect. Printing the top-level output for x.py --help check seems quite odd to me.

But if clap has this behavior, then I'm OK sticking with that decision (would be interesting to see if that's intentional). It's probably a rare case either way.

Yes, I checked that:

clap checks arguments from left to right

If an argument is help (--help or -h), it prints help of the current level (e.g., ./x --help is for top level, and ./x check --help is for check level) and exit immediately

So x.py --help check prints the top level help, and argument check is just ignored.

Mark-Simulacrum · 2025-10-11T18:47:07Z

r=me with commits squashed

Currently x.py help (or x.py --help) builds bootstrap binary everytime, but it delays printing help. This change saves the current top level help text into a file. x.py help prints the file and doesn't touch bootstrap binary. x.py test bootstrap checks if the file is up to date. Note that subcommand level helps (e.g., x.py check --help) aren't saved.

rustbot · 2025-10-11T22:21:48Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Mark-Simulacrum · 2025-10-11T22:29:38Z

@bors r+

bors · 2025-10-11T22:29:41Z

📌 Commit 340702c has been approved by Mark-Simulacrum

It is now in the queue for this repository.

…lacrum Save x.py's help text for saving output time Fix rust-lang#141903 Currently x.py help (--help) builds bootstrap binary everytime, so it takes some seconds to print help. This PR does: - Saves current help text into a file (x.py run generate-help) - Changes bootstrap.py to print the help in the saved file and to exit without touching bootstrap binary - Modifies x.py test bootstrap to check if the help file is up-to-date

Rollup of 12 pull requests Successful merges: - #138799 (core: simplify `Extend` for tuples) - #146692 (Save x.py's help text for saving output time) - #147168 (Don't unconditionally build alloc for `no-std` targets) - #147178 ([DebugInfo] Improve formatting of MSVC enum struct variants) - #147240 (Add an ACP list item to the library tracking issue template) - #147246 (Explain not existed key in BTreeMap::split_off) - #147393 (Extract most code from `define_feedable!`) - #147495 (Update wasm-component-ld to 0.5.18) - #147503 (Fix documentation of Instant::now on mac) - #147541 (Change int-to-ptr transmute lowering back to inttoptr) - #147549 (Replace `LLVMRustContextCreate` with normal LLVM-C API calls) - #147596 (Adjust the Arm targets in CI to reflect latest changes) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 8 pull requests Successful merges: - #138799 (core: simplify `Extend` for tuples) - #145897 (Rehome 30 `tests/ui/issues/` tests to other subdirectories under `tests/ui/` [#4 of Batch #2]) - #146692 (Save x.py's help text for saving output time) - #147240 (Add an ACP list item to the library tracking issue template) - #147246 (Explain not existed key in BTreeMap::split_off) - #147393 (Extract most code from `define_feedable!`) - #147503 (Fix documentation of Instant::now on mac) - #147549 (Replace `LLVMRustContextCreate` with normal LLVM-C API calls) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #146692 - Shunpoco:issue-141903, r=Mark-Simulacrum Save x.py's help text for saving output time Fix #141903 Currently x.py help (--help) builds bootstrap binary everytime, so it takes some seconds to print help. This PR does: - Saves current help text into a file (x.py run generate-help) - Changes bootstrap.py to print the help in the saved file and to exit without touching bootstrap binary - Modifies x.py test bootstrap to check if the help file is up-to-date

Shunpoco force-pushed the issue-141903 branch from 8fedea1 to 2e25304 Compare September 17, 2025 22:25

This comment has been minimized.

Sign in to view

Shunpoco force-pushed the issue-141903 branch from 2e25304 to 6c09bbb Compare September 17, 2025 22:46

Shunpoco changed the title ~~[WIP] Save x.py's help text for saving output time~~ Save x.py's help text for saving output time Sep 17, 2025

Shunpoco marked this pull request as ready for review September 17, 2025 23:48

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 17, 2025

rustbot assigned Mark-Simulacrum Sep 17, 2025

This comment has been minimized.

Sign in to view

Mark-Simulacrum reviewed Sep 28, 2025

View reviewed changes

Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 28, 2025

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 1, 2025

Mark-Simulacrum reviewed Oct 1, 2025

View reviewed changes

Shunpoco force-pushed the issue-141903 branch from 2464757 to c923756 Compare October 11, 2025 22:10

This comment has been minimized.

Sign in to view

Shunpoco force-pushed the issue-141903 branch from c923756 to 340702c Compare October 11, 2025 22:21

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 11, 2025

Zalathar mentioned this pull request Oct 12, 2025

Rollup of 12 pull requests #147598

Closed

matthiaskrgr mentioned this pull request Oct 12, 2025

Rollup of 8 pull requests #147603

Merged

bors merged commit f64f890 into rust-lang:master Oct 12, 2025
10 checks passed

rustbot added this to the 1.92.0 milestone Oct 12, 2025

Shunpoco deleted the issue-141903 branch October 12, 2025 16:37

Save x.py's help text for saving output time #146692

Save x.py's help text for saving output time #146692

Conversation

Shunpoco commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

rustbot commented Sep 17, 2025

Uh oh!

rustbot commented Sep 17, 2025

Uh oh!

jieyouxu commented Sep 18, 2025

Uh oh!

Kobzol commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Shunpoco commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum commented Sep 21, 2025

Uh oh!

jyn514 commented Sep 23, 2025

Uh oh!

Shunpoco commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Mark-Simulacrum Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

Shunpoco Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Shunpoco commented Oct 1, 2025

Uh oh!

Mark-Simulacrum Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

Shunpoco Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shunpoco Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum Oct 11, 2025

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum Oct 11, 2025

Choose a reason for hiding this comment

Uh oh!

Shunpoco Oct 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum commented Oct 11, 2025

Uh oh!

This comment has been minimized.

rustbot commented Oct 11, 2025

Uh oh!

Mark-Simulacrum commented Oct 11, 2025

Uh oh!

bors commented Oct 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Shunpoco commented Sep 17, 2025 •

edited

Loading

Kobzol commented Sep 18, 2025 •

edited

Loading

Shunpoco commented Sep 18, 2025 •

edited

Loading

Shunpoco commented Sep 23, 2025 •

edited

Loading

Shunpoco Oct 1, 2025 •

edited

Loading

Shunpoco Oct 11, 2025 •

edited

Loading