bootstrap: Overhaul and simplify the `tool_extended!` macro #135014

Zalathar · 2025-01-02T07:24:05Z

Similar to #134950, but for the macro that declares build steps for some tools.

The main changes are:

Removing some functionality that isn't needed by any of the tools currently using the macro
Moving some code out of the macro and into ordinary helper functions
Switching to one macro invocation per tool, and struct-like syntax so that rustfmt will format them

There should be no functional change.

This field was introduced in rust-lang#48097 to support the "clippy" feature of RLS.

rustbot · 2025-01-02T07:24:14Z

rustbot has assigned @albertlarsan68.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Using struct-like syntax allows rustfmt to format macro invocations, instead of giving up and ignoring them. Using a separate macro invocation per tool makes the macro slightly simpler, and isolates syntax errors to individual invocations.

Zalathar · 2025-01-02T07:44:27Z

Appeased clippy (diff).

jieyouxu

Thanks, this is a nice simplification

jieyouxu · 2025-01-02T10:00:01Z

@bors r+ rollup

bors · 2025-01-02T10:00:05Z

📌 Commit 5a32a35 has been approved by jieyouxu

It is now in the queue for this repository.

bors · 2025-01-02T11:33:49Z

⌛ Testing commit 5a32a35 with merge 70a3189...

bootstrap: Overhaul and simplify the `tool_extended!` macro Similar to rust-lang#134950, but for the macro that declares build steps for some tools. The main changes are: - Removing some functionality that isn't needed by any of the tools currently using the macro - Moving some code out of the macro and into ordinary helper functions - Switching to one macro invocation per tool, and struct-like syntax so that rustfmt will format them There should be no functional change.

rust-log-analyzer · 2025-01-02T13:27:35Z

The job dist-x86_64-msvc failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[2025-01-02T13:18:35Z DEBUG collector::compile::benchmark] Benchmark iteration 1/1
[2025-01-02T13:18:35Z INFO  collector::compile::execute] run_rustc with incremental=false, profile=Debug, scenario=Some(Full), patch=None, backend=Llvm, phase=benchmark
[2025-01-02T13:18:35Z DEBUG collector::compile::execute] "\\\\?\\C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "rustc" "--manifest-path" "Cargo.toml" "-p" "path+file:///C:/a/_temp/msys64/tmp/.tmprgXbYp#[email protected]" "--" "--wrap-rustc-with" "Eprintln"
[2025-01-02T13:18:36Z INFO  collector::compile::execute] run_rustc with incremental=true, profile=Debug, scenario=Some(IncrFull), patch=None, backend=Llvm, phase=benchmark
[2025-01-02T13:18:36Z DEBUG collector::compile::execute] "\\\\?\\C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "rustc" "--manifest-path" "Cargo.toml" "-p" "path+file:///C:/a/_temp/msys64/tmp/.tmprgXbYp#[email protected]" "--" "--wrap-rustc-with" "Eprintln" "-C" "incremental=C:\\a\\_temp\\msys64\\tmp\\.tmprgXbYp\\incremental-state"
[2025-01-02T13:18:38Z INFO  collector::compile::execute] run_rustc with incremental=true, profile=Debug, scenario=Some(IncrUnchanged), patch=None, backend=Llvm, phase=benchmark
[2025-01-02T13:18:38Z DEBUG collector::compile::execute] "\\\\?\\C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "rustc" "--manifest-path" "Cargo.toml" "-p" "path+file:///C:/a/_temp/msys64/tmp/.tmprgXbYp#[email protected]" "--" "--wrap-rustc-with" "Eprintln" "-C" "incremental=C:\\a\\_temp\\msys64\\tmp\\.tmprgXbYp\\incremental-state"
[2025-01-02T13:18:39Z DEBUG collector::compile::benchmark] Benchmark iteration 1/1
[2025-01-02T13:18:39Z INFO  collector::compile::execute] run_rustc with incremental=false, profile=Opt, scenario=Some(Full), patch=None, backend=Llvm, phase=benchmark
[2025-01-02T13:18:39Z DEBUG collector::compile::execute] "\\\\?\\C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "rustc" "--manifest-path" "Cargo.toml" "-p" "path+file:///C:/a/_temp/msys64/tmp/.tmpMM70Cq#[email protected]" "--release" "--" "--wrap-rustc-with" "Eprintln"
[2025-01-02T13:18:40Z INFO  collector::compile::execute] run_rustc with incremental=true, profile=Opt, scenario=Some(IncrFull), patch=None, backend=Llvm, phase=benchmark
---
   Compiling rustc_driver v0.0.0 (C:\a\rust\rust\compiler\rustc_driver)
[RUSTC-TIMING] rustc_driver test:false 4.882
error: linking with `link.exe` failed: exit code: 1104
  |
  = note: "C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\symbols.o" "<1 object files omitted>" "C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\deps\\rustc_driver-d9b8f1ae614bc28b.dll.lib" "C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib/{libcompiler_builtins-5fe297e6720ad64e.rlib}" "psapi.lib" "shell32.lib" "ole32.lib" "uuid.lib" "advapi32.lib" "ws2_32.lib" "ntdll.lib" "kernel32.lib" "advapi32.lib" "ole32.lib" "oleaut32.lib" "advapi32.lib" "cfgmgr32.lib" "gdi32.lib" "kernel32.lib" "msimg32.lib" "opengl32.lib" "user32.lib" "winspool.lib" "bcrypt.lib" "advapi32.lib" "legacy_stdio_definitions.lib" "kernel32.lib" "kernel32.lib" "advapi32.lib" "ntdll.lib" "userenv.lib" "ws2_32.lib" "dbghelp.lib" "/defaultlib:libcmt" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\advapi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-errorhandling-l1-1-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-file-fromapp-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-handle-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-ioring-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-4.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-5.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-6.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-7.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-memory-l1-1-8.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-synch-l1-2-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-sysinfo-l1-2-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-sysinfo-l1-2-3.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-sysinfo-l1-2-4.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-sysinfo-l1-2-6.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-util-l1-1-1.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-winrt-error-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-winrt-l1-1-0.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-core-wow64-l1-1-1.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\api-ms-win-security-base-l1-2-2.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\avrt.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\bcp47mrm.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\bcryptprimitives.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\clfsw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\dbghelp.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\elscore.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\gdi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\icu.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\imagehlp.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\kernel32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\ktmw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\netapi32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\normaliz.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\ntdll.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\ntdllk.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\ole32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\oleacc.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\oleaut32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\propsys.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\psapi.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\rtworkq.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\txfw32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\user32.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\usp10.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\version.dll_imports_indirect.lib" "C:\\a\\_temp\\msys64\\tmp\\rustc0sEbS5\\wofutil.dll_imports_indirect.lib" "/NXCOMPAT" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\stacker-4832cbf20a1ff3c8\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\psm-5d408f36ff041b80\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\blake3-3ec3f5d7ebc70701\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\blake3-3ec3f5d7ebc70701\\out" "/LIBPATH:C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.42.34433\\atlmfc\\lib\\x64" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\build\\rustc_llvm-17812f01546fee56\\out" "/LIBPATH:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\llvm\\lib" "/OUT:C:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage1-rustc\\x86_64-pc-windows-msvc\\release\\deps\\rustc_main-ccdec2f115cfff29.exe" "/OPT:REF,ICF" "/DEBUG" "/PDBALTPATH:%_PDB%" "/MANIFEST:EMBED" "/MANIFESTINPUT:C:\\a\\rust\\rust\\compiler\\rustc\\Windows Manifest.xml" "/WX"
  = note: LINK : fatal error LNK1104: cannot open file 'C:\a\rust\rust\build\x86_64-pc-windows-msvc\stage1-rustc\x86_64-pc-windows-msvc\release\deps\rustc_main-ccdec2f115cfff29.exe'␍
          

[RUSTC-TIMING] rustc_main test:false 0.672

bors · 2025-01-02T13:28:31Z

💔 Test failed - checks-actions

jieyouxu · 2025-01-02T13:28:41Z

@bors retry (msvc link.exe spurious failure)

bors · 2025-01-02T16:18:32Z

⌛ Testing commit 5a32a35 with merge 1b2745d...

bors · 2025-01-02T19:01:33Z

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 1b2745d to master...

rust-timer · 2025-01-02T20:17:52Z

Finished benchmarking commit (1b2745d): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 1.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[1.0%, 1.0%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	1.0%	[1.0%, 1.0%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 763.315s -> 762.677s (-0.08%)
Artifact size: 325.45 MiB -> 325.50 MiB (0.01%)

Zalathar added 3 commits January 2, 2025 17:47

Remove the unused extra_features field from tool_extended!

e76c484

This field was introduced in rust-lang#48097 to support the "clippy" feature of RLS.

Remove two unused parameters from tool_extended!

b84c54a

Don't pass (self, builder) identifiers to tool_extended!

bba24a2

rustbot assigned albertlarsan68 Jan 2, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jan 2, 2025

jieyouxu assigned jieyouxu and unassigned albertlarsan68 Jan 2, 2025

This comment has been minimized.

Sign in to view

Zalathar added 3 commits January 2, 2025 18:43

Move most of Step::run out of tool_extended!

725fccd

Use struct-like syntax in tool_extended!

08d70f5

Using struct-like syntax allows rustfmt to format macro invocations, instead of giving up and ignoring them. Using a separate macro invocation per tool makes the macro slightly simpler, and isolates syntax errors to individual invocations.

Move most of Step::should_run out of tool_extended!

5a32a35

Zalathar force-pushed the tool-build-step branch from 639b4d7 to 5a32a35 Compare January 2, 2025 07:43

jieyouxu approved these changes Jan 2, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 2, 2025

jieyouxu added the CI-spurious-fail-msvc CI spurious failure: target env msvc label Jan 2, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 2, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 2, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 2, 2025

bors merged commit 1b2745d into rust-lang:master Jan 2, 2025
7 checks passed

rustbot added this to the 1.85.0 milestone Jan 2, 2025

Zalathar deleted the tool-build-step branch January 2, 2025 23:07

jieyouxu mentioned this pull request Jan 4, 2025

bootstrap: correctly handle doc paths within submodules #135096

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bootstrap: Overhaul and simplify the `tool_extended!` macro #135014

bootstrap: Overhaul and simplify the `tool_extended!` macro #135014

Uh oh!

Zalathar commented Jan 2, 2025

Uh oh!

rustbot commented Jan 2, 2025

Uh oh!

This comment has been minimized.

Zalathar commented Jan 2, 2025

Uh oh!

jieyouxu left a comment

Uh oh!

jieyouxu commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

rust-log-analyzer commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

jieyouxu commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

Uh oh!

rust-timer commented Jan 2, 2025

Uh oh!

Uh oh!

bootstrap: Overhaul and simplify the tool_extended! macro #135014

bootstrap: Overhaul and simplify the tool_extended! macro #135014

Uh oh!

Conversation

Zalathar commented Jan 2, 2025

Uh oh!

rustbot commented Jan 2, 2025

Uh oh!

This comment has been minimized.

Zalathar commented Jan 2, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

rust-log-analyzer commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

jieyouxu commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

bors commented Jan 2, 2025

Uh oh!

Uh oh!

rust-timer commented Jan 2, 2025

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Uh oh!

bootstrap: Overhaul and simplify the `tool_extended!` macro #135014

bootstrap: Overhaul and simplify the `tool_extended!` macro #135014