Change how LLD is configured in bootstrap.toml

[Kobzol]

Currently, the way LLD (and using it as a default linker for a given target) is configured in bootstrap is a big mess. We have:

• rust.use-lld, which tells bootstrap to use LLD when building local Rust components, so it is only relevant for bootstrapping and faster local (re)builds.
• rust.lld, which both:
  • Tells bootstrap to build LLD and include it in the built rustc's sysroot
  • Tells rustc to use rust-lld by default (ONLY on x86_64-unknown-linux-gnu)
• rust.default-linker, which changes the default linker to be used by rustc, but this does not work on Linux, which is why we used the rust.lld hack above

I would suggest the following, based on prior discussions in #146604 and other PRs:

• Rename rust.use-lld to e.g. rust.bootstrap-override-lld, to make it clear that it's a bootstrapping-only flag.
  • Rename rust.use-lld to rust.bootstrap-override-lld #147046
• Move rust.lld to llvm.lld, because the LLVM section configures which LLVM-adjacent things we build, and make it only mean "build LLD and add it to sysroot", nothing else
• Add target.<target>.default-linker-linux, which can be either unset or "self-contained-lld-cc". It will be mutually exclusive with rust.default_linker.
  • Generalize configuring LLD as the default linker in bootstrap #147157

We should keep the old options for some time, to keep backwards compatibility.

[jyn514]

• Rename rust.use-lld to e.g. rust.bootstrap-with-lld, to make it clear that it's a bootstrapping-only flag.
• Move rust.lld to llvm.lld, because the LLVM section configures which LLVM-adjacent things we build, and make it only mean "build LLD and add it to sysroot", nothing else

👍

• Add rust.default_linux_linker (and .default_linux_linker), which can be either unset or "self-contained-lld". It will be mutually exclusive with rust.default_linker.

the docs for default-linker say this:

  # The default linker that will be hard-coded into the generated
  # compiler for targets that don't specify a default linker explicitly
  # in their target specifications.  Note that this is not the linker
  # used to link said compiler. It can also be set per-target (via the
  # `[target.<triple>]` block), which may be useful in a cross-compilation
  # setting.
  #
  # See https://doc.rust-lang.org/rustc/codegen-options/index.html#linker for more information.
  #rust.default-linker = <none> (path)

The behavior makes sense but it is not what I expect from the name. I think we should rename this to target.<triple>.fallback-linker, and disallow it at the top-level. Then we can make your new option named override_default_linker. I don't think linux is at all related here and I don't think it should be part of the option name.

[Kobzol]

I guess that we can avoid the Linux name in the option if we treat the overridden values as a hard-coded set of options that will be translated into specific code executed in the target configuration machinery.

So for now, we would allow only override_default_linker = "self-contained-lld" (and nothing else), and that would set

base.linker_flavor = LinkerFlavor::Gnu(Cc::Yes, Lld::Yes);
base.link_self_contained = crate::spec::LinkSelfContainedDefault::with_linker();

for the given target in the compiler. And if someone uses that on Windows, well, good luck 😆

[jyn514]

I think that's fine for now, yeah. You could imagine a smarter version of this that uses the appropriate LinkerFlavor for the target depending on what llvm.lld does, but I don't think we need that in the initial change, it's enough to error out on Windows.

[ChrisDenton]

What's the problem on Windows? Either we distribute a linker or not (in which case it'll be an error anyway). You can use lld in gcc mode on Windows. It'd be a bit odd with msvc targets but if you can somehow make it work than more power to you.

[Kobzol]

It's not really a problem, and I think that enabling usage of LLD on Windows would be great. It's just that the linker is configured differently; on Windows, lld is used directly as the linker, while on Linux, the linker is still cc, just with an additional flag that tells it to use LLD.

[mati865]

self-contained-lld might be a bit ambiguous. From the name, it's hard to tell whether that would mean fully self-contained (not relying on the CC driver) or just partially self-contained (being passed to the CC driver).

For windows-msvc this would always mean fully self-contained, for windows-gnu, well it's not so obvious.
Windows-gnu provides a component called rust-mingw which contains the necessary libraries and the linker (both GCC and ld.bfd). Now, when you link in self-contained mode (see https://doc.rust-lang.org/stable/nightly-rustc/rustc_target/spec/struct.LinkSelfContainedComponents.html for the options), that means you are using that shipped linker along with the libs.
In that case, should self-contained-lld mean full or partial self-contained mode for LLD?

Full disclosure, fully self-contained mode is not implemented for windows-gnu* yet, but I plan to add it to windows-gnullvm as described in #146634. If desired, it could be added to windows-gnu later.

On Linux, it's a clear situation right now, but if the linkers gain driverless mode (i.e., they would find the system libs on their own), it'd be a similar situation as with windows-gnu.
I'm not aware of linkers other than Wild (davidlattimore/wild#481) that plan to implement it, and whether #11937 is still valid. In short #11937 would make rustc call the linker directly with the set of the right system libraries, so somewhat similar to what windows-gnullvm would do.

[Kobzol]

To clarify, "self-contained-lld" (or whatever name we choose) would always invoke cc. If we wanted to use rust-lld as the linker directly (on non-Linux targets), there should be a separate option for that.

[Kobzol]

In other words, each such bootstrap option should directly translate to a specific block of code invoked in the target options machinery. It shouldn't depend on the target where it was applied, to avoid complicating the logic inside rustc. The individual options for LLD will be selected per-target in bootstrap.toml anyway.

[mati865]

To clarify, "self-contained-lld" (or whatever name we choose) would always invoke cc. If we wanted to use rust-lld as the linker directly (on non-Linux targets), there should be a separate option for that.

We know that, but will be as clear for the users? Fortunately, windows-gnu* is quite niche, so there are not that many people who could get confused 😆

[Kobzol]

We'll have to properly document it, of course :) I don't think that bootstrap implicitly using a different logic on Linux vs Windows targets with the same option value would be any better in this regard. Sadly the "linker situation" is very complicated, so we'll just have to document what exactly do we offer in bootstrap and rustc.