[GR-52516] How does GraalVM perform EXE linking on Windows?

[BullyWiiPlaza]

I noticed a bug caused by UPX when trying to compress GraalVM native EXEs: upx/upx#670

Currently, the maintainer is wondering how native-image exactly performs linking of the native EXE for Windows and what "unusual" things native-image might do which might break tools like UPX. Does GraalVM use LINK.exe from Visual C++ or only the SubstrateVM for linking? What's fundamentally different from native EXEs compiled with the Visual C++ toolchain compared to what native-image does in that regard? Is there any way to enable debug logging of linker commands? --verbose and/or --diagnostics-mode doesn't seem to do it. "Regular" native EXEs (compiled with Visual C++) certainly don't cause issues with UPX compression.

[adinn]

@BullyWiiPlaza Take a look for yourself. The code that links the image on Windows is defined in class WindowsCCLinkerInvocation (in package com.oracle.svm.hosted.image). Method getCommand() is responsible for constructing the link command. Obviously what gets into the command line depends on what you are building so reading that code will tell you definitively what options might arise.

The problem that is breaking UPX may very well not be down to a command option. The generated image includes various sections that are constructed from the Java code which are not going to look like the image sections generated by any Windows compiler. That doesn't mean they have an invalid format as far as Windows is concerned -- they certainly conform to the PECOFF standard -- but they may not be a format within that standard that UPX expects.

So, the 193 errors you are seeing probably reflect 193 cases where UPX does not expect or understand the binary layout GraalVM Native has chosen. What that implies as far as the program's correctness is concerned is probably nothing (although there may possibly be some layout bugs in there that don't break correct execution yet also fail conform 100% to PECOFF). What it implies as regards UPX's ability to compress the binary and what format it would be happy to consume instead is something that only the implementors of UPX can answer -- by inspecting the binary and seeing why it UPX does not like its layout.

n.b. The fact that UPX does not like the chosen format might be a reason for GraalVM Native to adopt a different one. I suspect that UPX is making some (simplifying) assumptions about the generated sections because it expects them to be generated by a Windows compiler. Unfortunately, GraalVM Native is pretty much guaranteed to produce image sections that don't look like that. So, even if GraalVM wanted to play ball with UPX it may not be possible to do so. Once again, this can only be confirmed by a UPX dev who undertstands how UPX operates explaining what is causing it to barf on the image layout.

[markus-oberhumer]

@adinn Many thanks for that info! UPX dev here

[petermz]

@BullyWiiPlaza as for the linker invocation, you can print the exact command line using -H:+TraceNativeToolUsage

[chirontt]

Some detailed technical discussion about this UPX problem was in an old issue #4340, especially about the WindowsImageHeapProvider.class (which then got moved to the WindowsFeature.class). Some reason for the problem was mentioned in upx/upx#559 (comment), which came from a comment from the original PR #4051 (comment).

But I don't know whether those discussions are relevant anymore for the latest GraalVM version.

[markus-oberhumer]

I did some debugging with x64dbg under Wine, but nothing unusual - the UPX compressed program simply exits with exit code 193.

INT3 breakpoint at <ntdll.RtlExitUserProcess> ([00006FFFFFF839C0](x64dbg://localhost/address64#00006FFFFFF839C0))!
Process stopped with exit code 0xC1 (193)

Does this exit code ring a bell for somebody?

[markus-oberhumer]

Update: found suspicious ERROR_BAD_EXE_FORMAT constant in Git source code

$  rg -w -i 0xC1

substratevm/src/com.oracle.svm.core.windows/src/com/oracle/svm/core/windows/WindowsImageHeapProvider.java
180:    private static final int ERROR_BAD_EXE_FORMAT = 0xC1;

[fernando-valdez]

Hello @BullyWiiPlaza, can you please confirm if the original issue (upx/upx#559) is still happening after building a native image using the latest version of GraalVM?

[BullyWiiPlaza]

Hello @BullyWiiPlaza, can you please confirm if the original issue (upx/upx#559) is still happening after building a native image using the latest version of GraalVM?

Hello, unfortunately, after updating all software, the issue still happens.

[sebthom]

Hello @BullyWiiPlaza, can you please confirm if the original issue (upx/upx#559) is still happening after building a native image using the latest version of GraalVM?

@fernando-valdez I can confirm that this issue (previously reported as #4340) still exists with latest GraalVM for JDK 22 Community 22.0.0. The last version that worked out of the box was GraalVM Community Edition 21.3.3.1 (Java 17) from September 2022.

See here for a real world reproducer: https://github.com/vegardit/copycat/actions/runs/8426676147/job/23075534217#check-step-13