rust 1.89.0 cross-build failure for aarch64_be (big-endian aarch64)

[he32]

I am doing the rounds of building rust 1.89.0 for the various NetBSD targets. I am building on NetBSD/amd64, and cross-building on that platform for the other targets. So far I have successfully built the rust compiler natively (of course), but also for armv6, armv7, powerpc, sparc64 and aarch64. However, when trying to build for big-endian aarch64 -- the aarch64_be-unknown-netbsd target, this fails with

   Compiling petgraph v0.6.5
   Compiling process-wrap v8.2.1
   Compiling zip v4.0.0
error[E0463]: can't find crate for `serde`
  --> /usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/vendor/serde_json-1.0.140/src/de.rs:16:5
   |
16 | use serde::de::{self, Expected, Unexpected};
   |     ^^^^^ can't find crate

error[E0463]: can't find crate for `serde`
  --> /usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/vendor/serde_json-1.0.140/src/de.rs:17:5
   |
17 | use serde::forward_to_deserialize_any;
   |     ^^^^^ can't find crate

error[E0463]: can't find crate for `serde`

and it's all downhill from there.

This failure is experienced during this phase of the build:

Building tool rust-analyzer (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)

which means that all the following phases (except the last one, of course, grep'ed from the build log) have successfully completed:

Building bootstrap
Building LLVM for aarch64_be-unknown-netbsd
Building LLVM for x86_64-unknown-netbsd
Building compiler artifacts (stage0 -> stage1, x86_64-unknown-netbsd)
Building stage0 tool lld-wrapper (x86_64-unknown-netbsd)
Building LLD for x86_64-unknown-netbsd
Building stage0 tool wasm-component-ld (x86_64-unknown-netbsd)
Building tool llvm-bitcode-linker (stage0 -> stage1, x86_64-unknown-netbsd)
Building stage1 library artifacts (x86_64-unknown-netbsd -> aarch64_be-unknown-netbsd)
Building stage1 library artifacts (x86_64-unknown-netbsd)
Building compiler artifacts (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Building stage1 tool lld-wrapper (x86_64-unknown-netbsd -> aarch64_be-unknown-netbsd)
Building LLD for aarch64_be-unknown-netbsd
Building stage1 tool wasm-component-ld (x86_64-unknown-netbsd -> aarch64_be-unknown-netbsd)
Building tool llvm-bitcode-linker (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Building tool rustdoc_tool_binary (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Building tool rust-analyzer-proc-macro-srv (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Building stage0 tool generate-copyright (x86_64-unknown-netbsd)
Building stage0 tool rust-installer (x86_64-unknown-netbsd)
Dist rustc-1.89.0-aarch64_be-unknown-netbsd
Dist rust-std-1.89.0-aarch64_be-unknown-netbsd
Dist rustc-dev-1.89.0-aarch64_be-unknown-netbsd
Dist rust-analysis-1.89.0-aarch64_be-unknown-netbsd
Dist rust-src-1.89.0
Building tool cargo (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Dist cargo-1.89.0-aarch64_be-unknown-netbsd
Building tool rust-analyzer (stage1:x86_64-unknown-netbsd -> stage2:aarch64_be-unknown-netbsd)
Build completed unsuccessfully in 1:55:16

I expected to see this happen: I expected the build to succeed, it certainly did for rust 1.88.0.

Instead, this happened: The build failed, as shown above.

I am a relative newbie to rust (yes, still...), and have a problem wrapping my head around why the choice of a particular target can have this effect, particularly when cross-building. It's all being built from the same tar-ball as distributed by the rust project.

Meta

Rust 1.89.0 as built natively is identified with:

rustc --version --verbose:

castor: {12} rustc --version --verbose
rustc 1.89.0 (29483883e 2025-08-04) (built from a source tarball)
binary: rustc
commit-hash: 29483883eed69d5fb4db01964cdf2af4d86e9cb2
commit-date: 2025-08-04
host: x86_64-unknown-netbsd
release: 1.89.0
LLVM version: 19.1.7
castor: {13}

However, the bootstrap build will use the upstream binary provided as part of rust 1.88.0.

Any hints pointing me in the direction of what might be going on here would be greatly appreciated.

[he32]

Just to update, in the mean time, the cross build for i386, riscv64 and mipsel have completed successfully.

[he32]

Question: does the big-endian aarch64 Linux target build? Cross-build or natively?

[Kobzol]

Does it work on master? We have been doing a lot of changes to bootstrap recently, and it's possible that it was fixed already (or also that one of the fixes actually broke this). I don't think that we would backport something like this, tbh.

[he32]

Does it work on master?

Sadly, I have no straight-forward way at the moment to answer that question. Our build setup is based on releases where all the vendored crates are integrated as part of the release tar-ball, and also relies on patching a number of crates (and therefore needing cargo checksum updates). In other words, preparing to build a new release is quite laborious.

Yes, it is probably high time for me to go through our patches and get those that would conceivably be accepted upstream be submitted for inclusion, so as to reduce the work related to preparing a new release, hopefully with effect some couple of releases forward.

I also hear mutterings about "we should consider setting up a package to build a 'beta' release", but so far that's not yet in place, so that's not available on a short time-scale.

We have been doing a lot of changes to bootstrap recently, [...]

"Uh-oh..." I'm bracing myself for the task of preparing a package for 1.90.0...

and it's possible that it was fixed already (or also that one of the fixes actually broke this). I don't think that we would backport something like this, tbh.

That's "sort of fine". Among all the rather "fringe" NetBSD targets, this is probably one of the "fringiest", so I'm not too worried about the real-world impact of this one not being build-able, especially if there is hope that it'll build in the future. We'll see.

I'll however still emphasize that I'm baffled that the choice of target to build for can make a difference in whether a vendored and definitely present crate is found or not during cross compilation while all the other NetBSD targets can successfully be cross-built on the same host using the same sources, and that this error occurs so late in the build process. "How can that even happen?!?" "Does this indicate a more serious problem?"

[he32]

Oh, yes, I also made an attempt at building rust 1.89.0 "natively" on an aarch64_be host, and that also failed, but "of course" with a different set of error messages, ref.:

   Compiling termize v0.1.1
   Compiling rustc_error_codes v0.0.0 (/usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/compiler/rustc_error_codes)
   Compiling termcolor v1.4.1
   Compiling rustc_feature v0.0.0 (/usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/compiler/rustc_feature)
   Compiling rustc_errors v0.0.0 (/usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/compiler/rustc_errors)
error: proc macro panicked
   --> compiler/rustc_errors/src/lib.rs:101:1
    |
101 | rustc_fluent_macro::fluent_messages! { "../messages.ftl" }
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: message: internal error: entered unreachable code

error[E0432]: unresolved import `crate::fluent_generated`
  --> compiler/rustc_errors/src/diagnostic_impls.rs:22:20
   |
22 |     Subdiagnostic, fluent_generated as fluent,
   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `fluent_generated` in the root

error[E0433]: failed to resolve: could not find `fluent_generated` in the crate root
   --> compiler/rustc_errors/src/diagnostic_impls.rs:392:10
    |
392 |   #[derive(Subdiagnostic)]
    |            ^^^^^^^^^^^^^
    |            |
    |            could not find `fluent_generated` in the crate root
    |            in this derive macro expansion
    |
   ::: /usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/vendor/synstructure-0.13.2/src/macros.rs:95:9
    |
95  | /         pub fn $derives(
96  | |             i: $crate::macros::TokenStream
97  | |         ) -> $crate::macros::TokenStream {
    | |________________________________________- in this expansion of `#[derive(Subdiagnostic)]`

error[E0433]: failed to resolve: could not find `fluent_generated` in the crate root
   --> compiler/rustc_errors/src/diagnostic_impls.rs:400:10
    |
400 |   #[derive(Subdiagnostic)]
    |            ^^^^^^^^^^^^^
    |            |
    |            could not find `fluent_generated` in the crate root
    |            in this derive macro expansion
    |
   ::: /usr/pkgsrc/wip/rust189/work/rustc-1.89.0-src/vendor/synstructure-0.13.2/src/macros.rs:95:9
    |
95  | /         pub fn $derives(
96  | |             i: $crate::macros::TokenStream
97  | |         ) -> $crate::macros::TokenStream {
    | |________________________________________- in this expansion of `#[derive(Subdiagnostic)]`

error[E0433]: failed to resolve: could not find `fluent_generated` in the crate root
    --> compiler/rustc_errors/src/lib.rs:1821:34
     |
1821 |                 let msg = crate::fluent_generated::errors_invalid_flushed_delayed_diagnostic_level;
     |                                  ^^^^^^^^^^^^^^^^ could not find `fluent_generated` in the crate root

error[E0433]: failed to resolve: could not find `fluent_generated` in the crate root
    --> compiler/rustc_errors/src/lib.rs:1863:49
     |
1863 |             BacktraceStatus::Captured => crate::fluent_generated::errors_delayed_at_with_newline,
     |                                                 ^^^^^^^^^^^^^^^^ could not find `fluent_generated` in the crate root

error[E0433]: failed to resolve: could not find `fluent_generated` in the crate root
    --> compiler/rustc_errors/src/lib.rs:1866:25
     |
1866 |             _ => crate::fluent_generated::errors_delayed_at_without_newline,
     |                         ^^^^^^^^^^^^^^^^ could not find `fluent_generated` in the crate root

   Compiling getopts v0.2.23
Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
error: could not compile `rustc_errors` (lib) due to 7 previous errors
warning: build failed, waiting for other jobs to finish...
Build completed unsuccessfully in 1 day, 13:06:25

This build error occcurs rather early among the build stages:

arm64eb: {2} egrep '^[BD]' list.01
Based on these variables, the following variables have been set:
Building bootstrap
Building LLVM for aarch64_be-unknown-netbsd
Building compiler artifacts (stage0 -> stage1, aarch64_be-unknown-netbsd)
Build completed unsuccessfully in 1 day, 13:06:25
arm64eb: {3} 

This is in contrast with the "normal" little-endian aarch64 NetBSD target where the "native" build succeeds (and the cross-build also succeeds for that target).

[Mark-Simulacrum]

Are you seeing any issues on non-big endian targets? I wonder if this is actually "just" some change that broke that... I'm not sure how much coverage mainline CI has (certainly we have no native big endian builds, I'm not sure we have any with host tools tier 2+ targets, but there's no easy index of that today I think).