--experimental_strict_action_env does not always enforce MSYS2 binaries

[filipesilva]

Description of the problem / feature request:

When using bazel from within Git Bash (aka Git for Windows) and using --experimental_strict_action_env, errors of the type This problem is probably due to using incompatible versions of the cygwin DLL. can still be seen.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

The same reproduction from #6339 can be used.

Instead of running bazel test ..., run bazel test ... --experimental_strict_action_env from a Git Bash shell.

What operating system are you running Bazel on?

Windows 10 1803.

What's the output of bazel info release?

release 0.19.2

Have you found anything relevant by searching the web?

This issue is related to #6339 and #5724.

/cc @laszlocsomor

[laszlocsomor]

Thanks for filing this! I managed to repro the issue like so (all commands in Git Bash):

1. git clone https://github.com/bazelbuild/bazel-integration-testing
2. cd bazel-integration-testing && git checkout 15273d8b0d01ce040800d172dbde3a88509d0c3d
3. export BAZEL_SH=c:/msys64/usr/bin/bash.exe
4. /c/src/bazel-releases/0.19.2/bazel.exe test --experimental_strict_action_env ...

This gives the error:

ERROR: error loading package '': Encountered error while reading extension file 'lib.bzl': no such package '@bazel_skylib//': Traceback (most recent call last):
        File "C:/_bazel/if4trwy3/external/bazel_tools/tools/build_defs/repo/git.bzl", line 166
                _clone_or_update(ctx)
        File "C:/_bazel/if4trwy3/external/bazel_tools/tools/build_defs/repo/git.bzl", line 72, in _clone_or_update
                fail(("error cloning %s:\n%s" % (ctx....)))
error cloning bazel_skylib:
+ cd C:/_bazel/if4trwy3/external
+ rm -rf C:/_bazel/if4trwy3/external/bazel_skylib C:/_bazel/if4trwy3/external/bazel_skylib
      5 [main] rm (19816) C:\git\usr\bin\rm.exe: *** fatal error - cygheap base mismatch detected - 0x1802FF410/0x180318410.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
ERROR: error loading package '': Encountered error while reading extension file 'lib.bzl': no such package '@bazel_skylib//': Traceback (most recent call last):
        File "C:/_bazel/if4trwy3/external/bazel_tools/tools/build_defs/repo/git.bzl", line 166
                _clone_or_update(ctx)
        File "C:/_bazel/if4trwy3/external/bazel_tools/tools/build_defs/repo/git.bzl", line 72, in _clone_or_update
                fail(("error cloning %s:\n%s" % (ctx....)))
error cloning bazel_skylib:
+ cd C:/_bazel/if4trwy3/external
+ rm -rf C:/_bazel/if4trwy3/external/bazel_skylib C:/_bazel/if4trwy3/external/bazel_skylib
      5 [main] rm (19816) C:\git\usr\bin\rm.exe: *** fatal error - cygheap base mismatch detected - 0x1802FF410/0x180318410.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
INFO: Elapsed time: 2.529s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Couldn't start the build. Unable to run tests
FAILED: Build did NOT complete successfully (0 packages loaded)

[laszlocsomor]

The problem is that --experimental_strict_action_env does not seem to affect repository rules. Even though the git repo rule respects BAZEL_SH and so runs the msys bash, it'll use the client environment which has the git bash directories on it, so the git bash rm is found instead of the msys bash rm.

[laszlocsomor]

I think repo rules should avoid using Bash commands as much as possible, and AFAIK @irengrig is working on a Bash-less version of the git_repository rule.

Bash-less repo rules are certainly our long-term goal. Windows users should not have to install MSYS. (Having to install Git is fine; Bazel ought not reimplement Git. However Git being bundled with MSYS is Git's own business that should not affect Bazel.)

I think it's the responsibility of repo rule owners to write Windows-safe (i.e. Bash-less) repo rules.
I can't tell if --experimental_strict_action_env should affect repo rules or not -- this would mean altering the client environment for repo rule actions, which the External Repositories team may have thoughts about. Assigning to External Repositories team.

@filipesilva : As a workaround, I recommend setting your PATH to have MSYS' directories before Git directories, and not installing git under MSYS. This way the repo rule will find all commands under MSYS except for Git. All of those should run in separate shells I believe, so you shouldn't see the DLL conflict error again. You could even create a wrapper script (Batch file?) to set up the environment for Bazel then run it.

[laszlocsomor]

@buchgr : What's your plan (if any) with --experimental_strict_action_env for Bazel 1.0? Do you plan to enable it by default, and to have repo rules respect it?