prefer dynamically built binaries when already installed

[2bndy5]

Its come to my attention that the statically built binaries may not be respecting the native libc++ distribution installed in the github runners. I think we should go back to relying on the pre-existing binaries that are already installed because these tend not to have problems finding C/C++ std libs.

Here is a flowchart that illustrates my thought process:

[shenxianpeng]

It looks good to do so. thanks for your flowchar!

[2bndy5]

I added this feature to the use-native-binaries branch, but the CI fails to create symlinks on the runner

When I tested this locally (with v12.0.1 installed via apt), it all went fine. Notice that I'm only checking that the major version matches. So, -i 12 will symlink to v12.0.0 or v12.0.1.

on the CI runner:

Found a installed version of clang-tidy: 12.0.0 at /usr/bin/clang-tidy-12

on my local machine (in WSL):

Found a installed version of clang-tidy: 12.0.1 at /usr/bin/clang-tidy-12

---

BTW, was there a reason to only run pytest on the main branch? I find it annoying to push something and then manually trigger the workflow every time (for any branch except main).

[shenxianpeng]

I didn't reproduce the CI fails on my environment, its clang-format location is /usr/lib/llvm-12/bin/clang-format, and it failed when linked to it.

It is OK just run pytest on the main branch and create a PR, so that it will not be triggered when git push to other branches.

[2bndy5]

Found a installed version of clang-tidy: 12.0.1 at /usr/bin/clang-tidy-12

its clang-format location is /usr/lib/llvm-12/bin/clang-format

I found that the /usr/bin/clang-tidy-12 was actually a symlink, so I made the prompt show the resolved path to the actual binary which is located in /usr/lib/llvm-12/bin/. Unfortunately, I can't figure out why the CI can't create a symlink to the binary.

As a quick hack, I guess we could make this pkg output the location of the binary to use instead of relying on a symlink in a known location. But the windows runner doesn't seem to be recognizing that v13 is already installed, so my implementation isn't ready for deployment.

[shenxianpeng]

Notice that I'm only checking that the major version matches. So, -i 12 will symlink to v12.0.0 or v12.0.1.

The CI fails may be related here. clang-tools find the exe_name with a major version, it's v12.0.0, so it did not download v12.0.1. but on Windows and macOS there is no v12, so it downloaded v12.0.1

clang-tools-pip/clang_tools/install.py

Line 33 in 1d42409

f"{tool_name}" + (f"-{ver_major}" if install_os != "windows" else "") + suffix

Then the CI runs which clang-format-12.0.1 it returns a non-zero value, so it failed.

[2bndy5]

I haven't said anything about this yet because I was hoping to get our fork of the static binaries repo setup properly.
We should really only be downloading the latest patched binaries and invoking them with only the major version number. So, -i 12 would download v12.0.1 and can be invoked with clang-*-12. It is odd that the Ubuntu runner ships with v12.0.0 instead of v12.0.1, but the LLVM project does seem to have a broad idea of what constitutes breaking changes

[shenxianpeng]

Yes, it's great if we can well maintain the fork binaries repo and fixed the existing issues.

[shenxianpeng]

We should really only be downloading the latest patched binaries and invoking them with only the major version number.

It looks good to me. The binaries repo provides both v12 and v12.0.1, so it also looks makes sense if users could download any version with clang-tools -i command.

[shenxianpeng]

I saw the following message in each action run when trying to create the symbolic link, this is because link directory does not exist when it trying to link to the target.

clang-tools-pip/clang_tools/install.py

Line 196 in 1d42409

link.symlink_to(target)

If we can create the dir before linking, the error will be gone. here is my test repo.

Found a installed version of clang-format: 12.0.0 at /usr/lib/llvm-12/bin/clang-format
Encountered an error when trying to create the symbolic link:
    No such file or directory
Found a installed version of clang-tidy: 12.0.0 at /usr/lib/llvm-12/bin/clang-tidy
Encountered an error when trying to create the symbolic link:
    No such file or directory

Found a installed version of clang-format: 10.0.0 at /usr/lib/llvm-10/bin/clang-format
Encountered an error when trying to create the symbolic link:
    No such file or directory
Found a installed version of clang-tidy: 10.0.0 at /usr/lib/llvm-10/bin/clang-tidy
Encountered an error when trying to create the symbolic link:
    No such file or directory

[2bndy5]

link is not directory. It is an absolute path to the dynamically linked binary which in this case is resolved() before being passed to create_symlink().

I suppose I could try asserting the strict parameter and see if that helps.

[2bndy5]

Found a installed version of clang-format: 12.0.0 at /usr/lib/llvm-12/bin/clang-format

The path seen at the end here is the value used as a target. the link path is created upon calling symlink_to()

[shenxianpeng]

link is not directory.

Sorry, it's install_dir does not exist.

clang-tools-pip/clang_tools/install.py

Line 174 in 1d42409

link = Path(install_dir) / (tool_name + suffix)

[2bndy5]

oh! I hate it when the problem is so simple and I missed it entirely. That should be an easy fix.