binaries fail strict validation on macOS

[mitchellh]

Zig Version

zig-linux-x86_64-0.11.0-dev.926+266e2e9a3

Steps to Reproduce and Observed Behavior

I'm unsure if you'll be able to reproduce this... I have no minimal reproduction but I can reliably reproduce with my software and I'm happy to pair sometime.

When updating from build 824 to build 926, with the identical build environment (Nix so everything is content hash identical), codesigning is now regularly failing for the past 3 days.

Last known working version: zig-linux-x86_64-0.11.0-dev.824+b3f4e0d09 (plus earlier releases for many months, I haven't had any issues)

Error below:

$ /usr/bin/codesign --force -s "$MACOS_CERTIFICATE_NAME" --options runtime zig-out/MyApp.app -v
1 identity imported.
keychain: "/Users/runner/Library/Keychains/build.keychain-db"
version: 512
class: 0x00000010 
attributes:
    0x00000000 <uint32>=0x00000010 
    0x00000001 <blob>="Mac Developer ID Application: Mitchell Hashimoto"
    0x00000002 <blob>=<NULL>
    0x00000003 <uint32>=0x00000001 
    0x00000004 <uint32>=0x00000000 
    0x00000005 <uint32>=0x00000000 
    0x00000006 <blob>=0x1DDCE3827A64403042576AAAB1E30016D665B4E9  "\035\334\343\202zd\326e\264\351"
    0x00000007 <blob>=<NULL>
    0x00000008 <blob>=0x7B38373139316361322D306663392D3131643430326235323132327D00  "{87191ca2-0fc9-11d4-849a}\000"
    0x00000009 <uint32>=0x0000002A  "\000\000\000*"
    0x0000000A <uint32>=0x00000800 
    0x0000000B <uint32>=0x00000800 
    0x0000000C <blob>=0x0000000000000000 
    0x0000000D <blob>=0x0000000000000000 
    0x0000000E <uint32>=0x00000001 
    0x0000000F <uint32>=0x00000001 
    0x00000010 <uint32>=0x00000001 
    0x00000011 <uint32>=0x00000000 
    0x00000012 <uint32>=0x00000001 
    0x00000013 <uint32>=0x00000001 
    0x00000014 <uint32>=0x00000001 
    0x00000015 <uint32>=0x00000001 
    0x00000016 <uint32>=0x00000001 
    0x00000017 <uint32>=0x00000001 
    0x00000018 <uint32>=0x00000001 
    0x00000019 <uint32>=0x00000001 
    0x0000001A <uint32>=0x00000001
zig-out/MyApp.app: main executable failed strict validation

(Note: I did muck with the output of this a little in this issue I didn't know if any of these bytes were secrets so if you want the original let me know -- only a few fields were modified)

The MyApp.app bundle is a mostly empty bundle that just has the zig-built binary as the entrypoint.

Expected Behavior

Codesign works since nothing has changed... all my dependency content hashes are identical and the app source was only changed for the field_type => type @typeInfo change in a recent Zig commit. Codesign from the last working version (zig-linux-x86_64-0.11.0-dev.824+b3f4e0d09) and prior versions was working for many months.

It feels like perhaps a linker change occurred which is making codesign upset?

[mitchellh]

Here are some more details, I don't know if they help:

YES WORKING (Zig build 824)

mitchellh@Mitchells-MacBook-Pro Downloads % codesign -dv --verbose=4 ~/Downloads/myapp.app/Contents/MacOS/myapp
Executable=/Users/mitchellh/Downloads/myapp.app/Contents/MacOS/myapp
Identifier=o/f9ff5c8312406d5cab80da04cc54dd90/myapp
Format=bundle with Mach-O thin (arm64)
CodeDirectory v=20400 size=57347 flags=0x20002(adhoc,linker-signed) hashes=1788+0 location=embedded
VersionPlatform=1
VersionMin=722688
VersionSDK=722688
Hash type=sha256 size=32
CandidateCDHash sha256=fe6d04dabd00d294151f9e4e56e664f8450454e2
CandidateCDHashFull sha256=fe6d04dabd00d294151f9e4e56e664f8450454e253946b870b57bf921e299ce6
Hash choices=sha256
CMSDigest=fe6d04dabd00d294151f9e4e56e664f8450454e253946b870b57bf921e299ce6
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=23920640
Executable Segment flags=0x1
Page size=16384
Launch Constraints:
	None
CDHash=fe6d04dabd00d294151f9e4e56e664f8450454e2
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

mitchellh@Mitchells-MacBook-Pro Downloads % codesign --force -s "Developer ID Application: Mitchell Hashimoto (ABCD1234)" --options runtime myapp.app -v
myapp.app: replacing existing signature
myapp.app: signed bundle with Mach-O thin (arm64) [com.mitchellh.myapp]

NOT WORKING (Zig build 926)

mitchellh@Mitchells-MacBook-Pro Downloads % codesign -dv --verbose=4 ~/Downloads/myapp.app/Contents/MacOS/myapp
Executable=/Users/mitchellh/Downloads/myapp.app/Contents/MacOS/myapp
Identifier=myapp
Format=bundle with Mach-O thin (arm64)
CodeDirectory v=20400 size=57312 flags=0x20002(adhoc,linker-signed) hashes=1788+0 location=embedded
VersionPlatform=1
VersionMin=722688
VersionSDK=722688
Hash type=sha256 size=32
CandidateCDHash sha256=e370f4d96dde7ded987cd168fcb8200565954c6f
CandidateCDHashFull sha256=e370f4d96dde7ded987cd168fcb8200565954c6f1f1d45a3ed12e4f954d011ee
Hash choices=sha256
CMSDigest=e370f4d96dde7ded987cd168fcb8200565954c6f1f1d45a3ed12e4f954d011ee
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=23920640
Executable Segment flags=0x1
Page size=16384
Launch Constraints:
	None
CDHash=e370f4d96dde7ded987cd168fcb8200565954c6f
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

mitchellh@Mitchells-MacBook-Pro Downloads % codesign --force -s "Developer ID Application: Mitchell Hashimoto (ABCD1234)" --options runtime myapp.app -v
myapp.app: replacing existing signature
myapp.app: main executable failed strict validation

[mitchellh]

I can easily get a signable binary and non-signable binary from identical source just changing the Zig version so let me know if you want me to run any object inspection commands on them (i.e. otool) to get more information to figure out their differences...

I can't share the binary publicly because its a private project :( but if @kubkon or someone wants to hop on Discord or something I'd be happy to share if that helps.

[kubkon]

The title is a little misleading as it suggests we fail to generate a valid codesignature, when instead it is a non-conformance to Apple tooling problem. If you don't mind, I will update the issue title, and I'm on it. I have a pretty good idea when it went wrong.

[kubkon]

FYI, here's a minimal repro:

❯ echo "pub fn main() void {}" > main.zig 
❯ zig build-exe main.zig                           
❯ codesign -f -s - main 
main: replacing existing signature
main: main executable failed strict validation

[kubkon]

Fix provided in #14049 It was my typo due to recent refactoring of the linker introduced in #13964 As you probably realised by now, Apple's tooling ain't great in pointing out what is wrong with the binary except that it failed strict validation, so for that reason I've added a smoke test which should hopefully detect this type of errors. Anyhow, the problem was that the __LINKEDIT load command didn't include the code signature blob in its size calculation and hence codesign refusing to touch the binary (FYI, the refusal actually happened in codesign_allocate which without this info was unable to deduce how to swap out code signature blobs).