GraalVM gate CI workflow failing to execute `mx --primary-suite-path substratevm native-image --help`

[zakkak]

Describe the issue
It looks like 6eee91e broke the "GraalVM gate" workflow which is now failing with:

Can not find /home/runner/work/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_CE0C781D19_JAVA11/graalvm-ce0c781d19-java11-21.2.0-dev/bin/native-image
Did you forget to build? Try `mx build`

Steps to reproduce the issue

To reproduce locally:

cd /tmp
git clone git@github.com:graalvm/mx.git --depth 1
git clone git@github.com:oracle/graal.git --depth 1
cd graal
../mx/mx --primary-suite-path substratevm native-image --help

Describe GraalVM and your environment:

• GraalVM version: 2b3fc59
• JDK major version: 11
• OS: Fedora 34
• Architecture: AMD64

[olpaw]

Hi @zakkak do you have a link to recent gates failing because of the changes in 6eee91e ?

As far as I can see the github CI builders that build images seem to work correctly atm. E.g.
https://github.com/oracle/graal/runs/2827894929 builds images just fine.

[olpaw]

Also note that in https://github.com/oracle/graal/blob/master/substratevm/SubstrateVM.md#building-images we explicitly state that
mx build needs to be run before you can use mx native-image. So if mx native-image accidentally worked without, it was just a coincidence you should not rely on.

[rschatz]

I'm currently looking into the other failure and I just stumbled over it: https://github.com/oracle/graal/runs/2830086066

Not sure what this check is actually trying to do. If WITHOUT_VCS is set, instead of the gate command, it's just running native-image --help, without building anything? That looks fishy.

@zakkak do you remember why you introduced that? Shouldn't we still just run the normal mx gate command after the rm -rf .git? fc6ef77

[rschatz]

I have added an attempted fix to my PR: #3478

[zakkak]

So if mx native-image accidentally worked without, it was just a coincidence you should not rely on.

That's interesting. Indeed it used to work, for quiet some time :). We are not really relying on that so it's not something we are interested in testing, please see below the reason why it was initially chosen.

@zakkak do you remember why you introduced that? Shouldn't we still just run the normal mx gate command after the rm -rf .git? fc6ef77

From: #2738 (comment) (that was hard to find :) )

Looking at the changes in 05d53b0 one sees that only 3 suites are affected (sdk, compiler, substratevm).
Now in these suites only a subset of the commands are affected. For instance in substratevm the only affected commmands are native-image and native-image-configure (the ones invoking _ensure_vm_built. As a result,
mx --primary-suite-path substratevm gate --tags build works fine even without this PR.

In order to test all the changes of this PR one needs to run:

mx --primary-suite-path substratevm native-image -version
mx --primary-suite-path compiler makegraaljdk /tmp/graaljdk
mx --primary-suite-path sdk javadoc

but this still allows people to perform changes that could break mx commands performed outside a vcs repository.
E.g. mx --primary-suite-path substratevm build

Unfortunately we cannot (it would take too long) test every mx command of graal, so I just settled with mx native-image.

[olpaw]

Not a bug. See
https://github.com/oracle/graal/pull/3478/files/a66ac449d026192b531b27ca734deebba28164c4..f8da81db53e73461a0091390f11e4f3ce5db49ea#r652440510
#3478 (comment)