Skip to content

./x check compiler/rust miri checks compiler crates twice, does not get cached #146502

@RalfJung

Description

@RalfJung

Summary

./x check compiler/rust miri checks compiler crates twice, and re-running the command without any source changes keeps re-checking a good chunk of the compiler.

Command used

./x check compiler/rustc miri

Actual behaviour

$ ./x check compiler/rustc miri
Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.03s
Checking stage1 compiler artifacts{rustc-main} (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Checking rustc_llvm v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_llvm)
    Checking rustc_lint v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_lint)
    Checking rustc_ty_utils v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_ty_utils)
    Checking rustc_const_eval v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_const_eval)
    Checking rustc_traits v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_traits)
    Checking rustc_sanitizers v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_sanitizers)
    Checking rustc_hir_analysis v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_hir_analysis)
    Checking rustc_codegen_ssa v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_codegen_ssa)
    Checking rustc_borrowck v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_borrowck)
    Checking rustc_privacy v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_privacy)
    Checking rustc_passes v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_passes)
    Checking rustc_codegen_llvm v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_codegen_llvm)
    Checking rustc_mir_build v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_mir_build)
    Checking rustc_hir_typeck v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_hir_typeck)
    Checking rustc_mir_transform v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_mir_transform)
    Checking rustc_interface v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_interface)
    Checking rustc_driver_impl v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_driver_impl)
    Checking rustc_driver v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_driver)
    Checking rustc-main v0.0.0 (/home/r/src/rust/rustc/compiler/rustc)
    Finished `release` profile [optimized + debuginfo] target(s) in 2.05s
Checking stage1 compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Checking rustc_llvm v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_llvm)
    Checking rustc_lint v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_lint)
    Checking rustc_ty_utils v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_ty_utils)
    Checking rustc_traits v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_traits)
    Checking rustc_sanitizers v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_sanitizers)
    Checking rustc_const_eval v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_const_eval)
    Checking rustc_hir_analysis v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_hir_analysis)
    Checking rustc_codegen_ssa v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_codegen_ssa)
    Checking rustc_borrowck v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_borrowck)
    Checking rustc_privacy v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_privacy)
    Checking rustc_passes v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_passes)
    Checking rustc_codegen_llvm v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_codegen_llvm)
    Checking rustc_mir_build v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_mir_build)
    Checking rustc_hir_typeck v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_hir_typeck)
    Checking rustc_mir_transform v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_mir_transform)
    Checking rustc_interface v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_interface)
    Checking rustc_driver_impl v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_driver_impl)
    Checking rustc_driver v0.0.0 (/home/r/src/rust/rustc/compiler/rustc_driver)
    Checking rustc-main v0.0.0 (/home/r/src/rust/rustc/compiler/rustc)
    Finished `release` profile [optimized + debuginfo] target(s) in 2.07s
Checking stage1 miri (stage0 -> stage1, x86_64-unknown-linux-gnu)
    Finished `release` profile [optimized + debuginfo] target(s) in 0.09s
Build completed successfully in 0:00:04
$ ./x check compiler/rustc miri
# (same output again)

Expected behaviour

"Check rustc..." should only show up at most once. And when I re-run the command, it shouldn't show up at all.

Bootstrap configuration (bootstrap.toml)

# Sample TOML configuration file for building Rust.
#
# To configure rustbuild, copy this file to the directory from which you will be
# running the build, and name it config.toml.
#
# All options are commented out by default in this file, and they're commented
# out with their default values. The build system by default looks for
# `config.toml` in the current directory of a build for build configuration, but
# a custom configuration file can also be specified with `--config` to the build
# system.

change-id = 125535

# =============================================================================
# Tweaking how LLVM is compiled
# =============================================================================
[llvm]

# Whether to use Rust CI built LLVM instead of locally building it.
#
# Unless you're developing for a target where Rust CI doesn't build a compiler
# toolchain or changing LLVM locally, you probably want to set this to true.
#
# It's currently false by default due to being newly added; please file bugs if
# enabling this did not work for you on x86_64-unknown-linux-gnu.
# Other target triples are currently not supported; see #77084.
#
# We also currently only support this when building LLVM for the build triple.
#
# Note that many of the LLVM options are not currently supported for
# downloading. Currently only the "assertions" option can be toggled.
download-ci-llvm = "if-unchanged"

# Indicates whether the LLVM build is a Release or Debug build
#optimize = true

# Indicates whether an LLVM Release build should include debug info
#release-debuginfo = false

# Indicates whether the LLVM assertions are enabled or not
#assertions = false

# Indicates whether ccache is used when building LLVM
#ccache = false
# or alternatively ...
#ccache = "/path/to/ccache"

# If an external LLVM root is specified, we automatically check the version by
# default to make sure it's within the range that we're expecting, but setting
# this flag will indicate that this version check should not be done.
#version-check = true

# Tell the LLVM build system to use Ninja instead of the platform default for
# the generated build system. This can sometimes be faster than make, for
# example.
ninja = true

# LLVM targets to build support for.
# Note: this is NOT related to Rust compilation targets. However, as Rust is
# dependent on LLVM for code generation, turning targets off here WILL lead to
# the resulting rustc being unable to compile for the disabled architectures.
# Also worth pointing out is that, in case support for new targets are added to
# LLVM, enabling them here doesn't mean Rust is automatically gaining said
# support. You'll need to write a target specification at least, and most
# likely, teach rustc about the C ABI of the target. Get in touch with the
# Rust team and file an issue if you need assistance in porting!
#targets = "X86"
#experimental-targets = ""

# Cap the number of parallel linker invocations when compiling LLVM.
# This can be useful when building LLVM with debug info, which significantly
# increases the size of binaries and consequently the memory required by
# each linker process.
# If absent or 0, linker invocations are treated like any other job and
# controlled by rustbuild's -j parameter.
#link-jobs = 0

# When invoking `llvm-config` this configures whether the `--shared` argument is
# passed to prefer linking to shared libraries.
#link-shared = false

# On MSVC you can compile LLVM with clang-cl, but the test suite doesn't pass
# with clang-cl, so this is special in that it only compiles LLVM with clang-cl
#clang-cl = '/path/to/clang-cl.exe'
# =============================================================================
# General build configuration options
# =============================================================================
[build]

# The default stage to use for the `check` subcommand
#check-stage = 1

# Reduce RAM consumption
jobs = 12

# Instead of downloading the src/stage0.json version of Cargo specified, use
# this Cargo binary instead to build all Rust code
# If you set this, you likely want to set `rustc` as well.
#cargo = "/home/r/src/rust/cargo/target/debug/cargo"

# Instead of downloading the src/stage0.json version of the compiler
# specified, use this rustc binary instead as the stage0 snapshot compiler.
# If you set this, you likely want to set `cargo` as well.
#rustc = "/path/to/rustc"

# Indicate whether git submodules are managed and updated automatically.
submodules = false

# Enable a build of the extended Rust tool set which is not only the compiler
# but also tools such as Cargo. This will also produce "combined installers"
# which are used to install Rust and Cargo together.
# The `tools` (check `config.example.toml` to see its default value) option specifies
# which tools should be built if `extended = true`.
#
# This is disabled by default.
#extended = false

# Set of tools to be included in the installation.
#
# If `extended = false`, the only one of these built by default is rustdoc.
#
# If `extended = true`, they're all included, with the exception of
# rust-demangler which additionally requires `profiler = true` to be set.
#
# If any enabled tool fails to build, the installation fails.
tools = [
#    "cargo",
    "miri",
    "clippy",
    "rustdoc",
    "rustfmt",
#    "rust-analyzer",
#    "analysis",
#    "src",
#    "rust-demangler",  # if profiler = true
]

# =============================================================================
# Options for compiling Rust code itself
# =============================================================================
[rust]

# Whether or not to optimize when compiling the compiler and standard library,
# and what level of optimization to use.
# WARNING: Building with optimize = false is NOT SUPPORTED. Due to bootstrapping,
# building without optimizations takes much longer than optimizing. Further, some platforms
# fail to build without this optimization (c.f. #65352).
# The valid options are:
# true - Enable optimizations.
# false - Disable optimizations.
# 0 - Disable optimizations.
# 1 - Basic optimizations.
# 2 - Some optimizations.
# 3 - All optimizations.
# "s" - Optimize for binary size.
# "z" - Optimize for binary size, but also turn off loop vectorization.
#optimize = true

# Indicates that the build should be configured for debugging Rust. A
# `debug`-enabled compiler and standard library will be somewhat
# slower (due to e.g. checking of debug assertions) but should remain
# usable.
#
# Note: If this value is set to `true`, it will affect a number of
#       configuration options below as well, if they have been left
#       unconfigured in this file.
#
# Note: changes to the `debug` setting do *not* affect `optimize`
#       above. In theory, a "maximally debuggable" environment would
#       set `optimize` to `false` above to assist the introspection
#       facilities of debuggers like lldb and gdb. To recreate such an
#       environment, explicitly set `optimize` to `false` and `debug`
#       to `true`. In practice, everyone leaves `optimize` set to
#       `true`, because an unoptimized rustc with debugging
#       enabled becomes *unusably slow* (e.g. rust-lang/rust#24840
#       reported a 25x slowdown) and bootstrapping the supposed
#       "maximally debuggable" environment (notably libstd) takes
#       hours to build.
#
#debug = false

# Whether to download the stage 1 and 2 compilers from CI.
# This is useful if you are working on tools, doc-comments, or library (you will be able to build
# the standard library without needing to build the compiler).
#
# Set this to "if-unchanged" to only download if the compiler (and library if running on CI) have
# not been modified.
# Set this to `true` to download unconditionally.
download-rustc = "if-unchanged"

# Whether or not debug assertions are enabled for the compiler and standard
# library. Debug assertions control the maximum log level used by rustc. When
# enabled calls to `trace!` and `debug!` macros are preserved in the compiled
# binary, otherwise they are omitted.
#
# Defaults to rust.debug value
# FIXME: should be 'true' but then it doesn't download rustc...
debug-assertions = false

# Whether or not debug assertions are enabled for the standard library.
# Overrides the `debug-assertions` option, if defined.
#
# Defaults to rust.debug-assertions value
debug-assertions-std = false

# Debuginfo level for most of Rust code, corresponds to the `-C debuginfo=N` option of `rustc`.
# `0` - no debug info
# `1` - line tables only - sufficient to generate backtraces that include line
#       information and inlined functions, set breakpoints at source code
#       locations, and step through execution in a debugger.
# `2` - full debug info with variable and type information
# Can be overridden for specific subsets of Rust code (rustc, std or tools).
# Debuginfo for tests run with compiletest is not controlled by this option
# and needs to be enabled separately with `debuginfo-level-tests`.
#
# Note that debuginfo-level = 2 generates several gigabytes of debuginfo
# and will slow down the linking process significantly.
#
# Defaults to 1 if debug is true
debuginfo-level = 1

# Whether or not `panic!`s generate backtraces (RUST_BACKTRACE)
#backtrace = true

# Whether to always use incremental compilation when building rustc
incremental = true

# Build a multi-threaded rustc
#parallel-compiler = false

# The default linker that will be hard-coded into the generated compiler for
# targets that don't specify linker explicitly in their target specifications.
# Note that this is not the linker used to link said compiler.
#default-linker = "cc"

# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features
#channel = "nightly"

# By default the `rustc` executable is built with `-Wl,-rpath` flags on Unix
# platforms to ensure that the compiler is usable by default from the build
# directory (as it links to a number of dynamic libraries). This may not be
# desired in distributions, for example.
#rpath = true

# Prints each test name as it is executed, to help debug issues in the test harness itself.
verbose-tests = true

# Flag indicating whether tests are compiled with optimizations (the -O flag).
#optimize-tests = true

# Flag indicating whether codegen tests will be run or not. If you get an error
# saying that the FileCheck executable is missing, you may want to disable this.
# Also see the target's llvm-filecheck option.
#codegen-tests = true

# Flag indicating whether git info will be retrieved from .git automatically.
# Having the git information can cause a lot of rebuilds during development.
# Note: If this attribute is not explicitly set (e.g. if left commented out) it
# will default to true if channel = "dev", but will default to false otherwise.
#ignore-git = true

# When creating source tarballs whether or not to create a source tarball.
#dist-src = false

# This is an array of the codegen backends that will be compiled for the rustc
# that's being compiled. The default is to only build the LLVM codegen backend,
# and currently the only standard options supported are `"llvm"`, `"cranelift"`
# and `"gcc"`. The first backend in this list will be used as default by rustc
# when no explicit backend is specified.
codegen-backends = ["llvm"]

# Indicates whether some LLVM tools, like llvm-objdump, will be made available in the
# sysroot.
#llvm-tools = false

# Whether to deny warnings in crates
deny-warnings = false

# Print backtrace on internal compiler errors during bootstrap
#backtrace-on-ice = false

# Whether to verify generated LLVM IR
#verify-llvm-ir = false

# Link the compiler against `jemalloc`, where on Linux and OSX it should
# override the default allocator for rustc and LLVM.
#jemalloc = false

# Run tests in various test suites with the "nll compare mode" in addition to
# running the tests in normal mode. Largely only used on CI and during local
# development of NLL
#test-compare-mode = false

# Use LLVM libunwind as the implementation for Rust's unwinder.
#llvm-libunwind = false

Operating system

Debian testing

HEAD

408eacf

Additional context

This seems to be a fairly recent regression -- the same command behaved entirely as expected until not too long ago.

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: This is a bug.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)needs-triageThis issue may need triage. Remove it if it has been sufficiently triaged.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions