Compilation pipeline: spawn Jobs earlier that produce linker inputs #22541
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rexicon226
reviewed
Jan 19, 2025
Member
Author
|
Alright I've narrowed this down to: SEGV: OK: Clearly the difference here is the ordering of crti.o and crtn.o, as I suspected. |
andrewrk
added
breaking
Member
Author
|
Technically breaking now since it drops support for legacy .ini/.fini mechanism. See the relevant commit for more details. |
This means doing more work in parallel which is already good, but it's also a correctnes fix because we need link_task_wait_group.wait() to ensure that no more linker inputs will be generated.
Move all the remaining Jobs that produce linker inputs to be spawned earlier in the pipeline and registered with link_task_wait_group.
crti.o/crtn.o is a legacy strategy for calling constructor functions upon object loading that has been superseded by the init_array/fini_array mechanism. Zig code depends on neither, since the language intentionally has no way to initialize data at runtime, but alas the Zig linker still must support this feature since popular languages depend on it. Anyway, the way it works is that crti.o has the machine code prelude of two functions called _init and _fini, each in their own section with the respective name. crtn.o has the machine code instructions comprising the exitlude for each function. In between, objects use the .init and .fini link section to populate the function body. This function is then expected to be called upon object initialization and deinitialization. This mechanism is depended on by libc, for example musl and glibc, but only for older ISAs. By the time the libcs gained support for newer ISAs, they had moved on to the init_array/fini_array mechanism instead. For the Zig linker, we are trying to move the linker towards order-independent objects which is incompatible with the legacy crti/crtn mechanism. Therefore, this commit drops support entirely for crti/crtn mechanism, which is necessary since the other commits in this branch make it nondeterministic in which order the libc objects and the other link inputs are sent to the linker. The linker is still expected to produce a deterministic output, however, by ignoring object input order for the purposes of symbol resolution.
Turns out that even modern Debian aarch64 glibc libc_nonshared.a has references to _init, meaning that the previous commit caused a regression when trying to build any -lc executable on that target. This commit backs out the changes to LibCInstallation. There is still a fork in the road coming up when the self-hosted ELF linker becomes load bearing on that target.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This means doing more work in parallel which is already good, but it's also a correctnes fix because we need link_task_wait_group.wait() to ensure that no more linker inputs will be generated.
A side effect of this is that the Elf linker now receives link inputs in a nondeterministic order, and that seems to cause problems in particular with musl libc, so I need to troubleshoot that before landing this fix.
Addresses problem introduced by #22220.