Use uvx for CMake and Server Toolchain for Ninja

[eramongodb]

Updates the Evergreen configuration and scripts to use uv to obtain CMake binaries. This removes the need for the find-cmake-old.sh, find-cmake-latest.sh (C Driver), and find-cmake-version.sh (C Driver) helper scripts in the C++ Driver.

Also updates the EVG configuration to unconditionally use the Ninja generator instead of Unix Makefiles on non-Windows distros. However, instead of using uv, the Server Toolchain is used instead. The reasons for this are explained below.

This PR is intended to be a "simpler" preview of a followup PR which will apply similar changes to the C Driver.

---

First, this PR updates the uv-installer.sh script to obtain the current latest release of uv, bumping the uv version from 0.5.14 (Jan 2025) to 0.8.0 0.8.2 (July 2025). Related checksums continue to be manually applied to the uv-installer.sh script. A new patch-uv-installer.sh script is used to automatically patch the installer script to enable and verify checksums. To assist with future updates, a Bash script which may be used to obtain the list of checksums is now documented as a comment within the installer script, copied here for reference:

for checksum in $(curl -sSL https://github.com/astral-sh/uv/releases/download/0.8.0/dist-manifest.json | jq -r '.releases[0].artifacts.[] | select(startswith("uv-") and (endswith(".zip.sha256") or endswith(".tar.gz.sha256")))'); do
  curl -sSL "https://github.com/astral-sh/uv/releases/download/0.8.0/${checksum}"
done

Update: the download_checksums helper function in patch-uv-installer.sh can be used to obtain the list of checksums for a given release.

Note

Downloading the checksums at runtime would be convenient, but would also defeat the point of the checksum validation as a guard against corrupt or malicious downloads since the checksum values would come from the same potentially-corrupted or potentially-malicious source as the artifacts to be validated.

The uv.lock file is also updated with latest package upgrades by running uv lock -U.

---

The bulk of this PR is updating the Evergreen configuration and scripts to use cmake binaries provided by uv instead of the system-provided binaries (inconsistent availability and versions) or manual download strategies (i.e. via find-cmake-latest.sh). This is expected to improve the reliability, consistency, and ease of build tool acquisition and usage across EVG distros when the same pattern can be applied to other Python-packaged tools going forward.

Update: for compatibility with uv 0.8.1 and newer, which implemented stronger protections against bypassing ephemeral environments in astral-sh/uv#14790 (which is exactly what the uv run ... "command -v <name>" pattern was doing), this PR now proposes using uv tool install with the UV_TOOL_DIR and UV_TOOL_BIN_DIR environment variables set to local directories. This should not be a significant performance penalty, as the uv cache directory would still be used to populate the custom tool directories. However, these environment variables unfortunately require the C:\path form on Windows instead of /cygdrive/c/path (otherwise, no diagnostic or error (?!); instead, ignored in favor of the user's local bin directory 😢).

Rather than uv tool install cmake (which may modify the user's environment, e.g. by installing binaries to $HOME/.local/bin), this PR proposes using uvx cmake instead (uvx is a convenient short-hand for uv tool run). Unlike uv tool install, uvx does not modify the user's environment. When a full path to the binary is required (e.g. for CMake compatibility tests or as an argument to other scripts/tools), the lengthy uv run --no-project --isolated --with cmake bash -c "command -v cmake" command is required. There unfortunately does not yet seem to be a short-hand uv tool command to obtain a path to the uv-managed executable tool. Note that uvx bash -c "..." does not work due to uvx interpreting bash as a Python package.

Note

Concerning flags to uv run:

• --no-project prevents discovery and installation (uv sync) of a project before running the command, but may still reuse a virtual environment that is active or found in the current or parent directories.
• --isolated prevents reuse of any virtual environment by always using an isolated virtual environment, but may discover and install (uv sync) a project within that isolated virtual environment before running the command.

Both flags are required to ensure an isolated virtual environment without any project detection and installation is used. However, neither flag prevent use of uv's thread-safe, append-only dependency cache, which is how the same <name> binary can be safely reused by direct reference as obtained via command -v <name> (unless someone unsafely modifies the cache via uv cache or direct filesystem operations).

Although uvx has an --isolated flag, it is currently redundant pending resolution of astral-uv/uv#7186. For now, the uvx command always uses an isolated virtual environment.

---

This PR initially attempted to use the same approach to obtain the ninja binary. Obtaining ninja via uv is simple; however, this method was greatly frustrated by unexpected integration issues. After much effort, I've given up using uv-provided Ninja binaries; instead, this PR proposes unconditionally using the binary provided by Server Toolchain v4, which is available on all our currently-used EVG distros.

The first complication is that there is no CMAKE_MAKE_PROGRAM environment variable to go alongside the CMAKE_GENERATOR environment variable. This means the path to the Ninja binary (which requires the lengthy uv run command described above) is almost always necessary and has to be explicitly passed as a CLI argument during the CMake configuration step via -D CMAKE_MAKE_PROGRAM=<path/to/ninja>, which is not automatically propogated to subprocess or other tools (e.g. distcheck target, compile-libmongocrypt.sh script, etc.). This ultimately ends up forcing the addition of the ninja binary's parent directory to the PATH environment variable for reliable and consistent behavior.

The second and far worse complication is a strange bug which only seems to manifest when all of the following conditions are met:

• OS: RHEL
• CMake version 3.28 or newer (as obtained via uv tool)
• Ninja (as obtained via uv tool; version?)
• CXX: clang++ (version?)
• CMake Generator: Ninja
• C++ Standard: 20 or newer

Given these conditions, CMake configuration fails with the following unexpected error for CMake 3.29 or newer:

CMake Error at FindPackageHandleStandardArgs.cmake:227 (message):
  Could NOT find Threads (missing: Threads_FOUND)

and with the following unexpected error for CMake 3.28:

/bin/sh: line 1: CMAKE_CXX_COMPILER_CLANG_SCAN_DEPS-NOTFOUND: command not found

I am completely baffled by this issue. My best guess is that there may be an issue with how either CMake or Ninja are being packaged for Python, as system-installed equivalents (for CMake or Ninja) do not appear to exhibit this issue. I could not manage to work out a way to patch this erroneous behavior that didn't end up becoming a variation of this terrible special-casing code across various EVG scripts:

ninja_binary="$(uv run --no-project --isolated --with ninja bash -c "command -v ninja")"

# Strange issue only on RHEL...
if [[ "${distro_id:?}" =~ rhel ]]; then
  : # ... when using PyPI-provided CMake (3.28 or newer), Clang, and Ninja...
  if [[ "${CXX:-}" =~ clang\+\+ && "${generator:-"Ninja"}" == Ninja ]]; then
    : # ... and when using C++20 or newer...
    if [[ "${REQUIRED_CXX_STANDARD:-}" -ge 20 ]]; then
      # ... results in `Could NOT find Threads (missing: Threads_FOUND)` (???).
      # Use Ninja provided by MongoDB Toolchain v4 as a workaround.
      ninja_binary=/opt/mongodbtoolchain/v4/bin/ninja
    fi
  fi
fi

command -v "${ninja_binary:?}"

Due to these two issues, I decided to unconditionally use the MongoDB Server Toolchain, as provided by the /opt/mongodbtoolchain/v4/bin path on all our currently-used non-Windows distros on RHEL distros only:

[[ "${distro_id:?}" == rhel* ]] && PATH="${PATH:-}:/opt/mongodbtoolchain/v4/bin" || uv tool install -q ninja

The v4 toolchain is chosen due to currently having better distro availability than v5 (e.g. it is missing on the rhel7-latest distro). In spite of prior efforts to avoid depending on the MongoDB Toolchain (due to Server Team not supporting any downstream users beyond themselves), I believe switching from Unix Makefiles to Ninja is nevertheless worth the effort. We are already (knowingly) depending on the Server Toolchain for other tasks (e.g. sanitizers and ccache) despite this lack of downstream support guarantee. I am in favor of actively maintaining compatibility with the Server Toolchain ourselves (as we've already been doing) rather than attempting workarounds like those described above. We can revisit obtaining ninja via uv in followup PRs (i.e. on Windows distros to finally have ccache-friendly MSVC builds).

Note

The path to the Server Toolchain is appended rather than prepended to PATH to avoid having higher precedence when invoking compilers (e.g. g++, clang++, etc.) and other binaries. It is okay if we end up using a system ninja (such as on Ubuntu distros): we just need to guarantee the availability of a ninja binary for use as the CMake generator without needing to also use the CMAKE_MAKE_PROGRAM configuration variable.

---

Miscellaneous drive-by fixes and improvements include:

• setup_group_can_fail_task (incorrect) -> setup_task_can_fail_task (correct) in the abi-stability task group definition.
• Use PEP 0440 syntax to specify CMake binary versions to use in cmake-compat tasks (e.g. cmake~=3.0 for "latest v3 minor release", cmake~=3.15.0 for "latest v3.15 patch release", etc.).
• Removal of extranneous commands in setup EVG function, which have been redundant/obsolete/unused information for a while now.
• Move the "uninstall" build command out of uninstall_check* scripts to avoid the need to handle paths within the scripts themselves (in particular the Windows batch script).
• Improve MSBuild parallelism by using UseMultiToolTask and EnforceProcessCountAcrossBuilds (VS 2019 16.3 and newer) and /maxcpucount CMAKE_BUILD_PARALLEL_LEVEL (CMake 3.26 and newer) in compile.sh.
• Extend ccache and CMake parallelism to building libmongocrypt in install-c-driver.sh.

[kevinAlbs]

The efforts to move towards uv and Ninja are much appreciated. LGTM with a fix to macOS builds.

Aside: I would like to avoid manually patching uv-installer.sh, but I do not see a clear alternative.

[eramongodb]

Updated the PR to include a new patch-uv-installer.sh script which automatically patches the uv-installer.sh script to enable checksum validation. The list of checksums to be validated with is embedded in the patch-uv-installer.sh script along with a download_checksums helper function which may be used to update the list of checksums.

Updated the PR to use uv tool install as designed and intended rather than the uv run --isolated ... pattern initially proposed. This is due to uv 0.8.1 and newer implementing stronger protections against bypassing ephemeral environments in astral-sh/uv#14790 (following astral-sh/uv#14447 in 0.8.0), which breaks the uv run --isolated ... pattern, as the directory no longer exists after the command as completed execution.

Instead, the UV_TOOL_DIR and UV_TOOL_BIN_DIR environment variables are used to install the required binaries into (task-)local directories. This should not be a significant performance problem, since the (user-level) uv cache will still be utilized to populate the local tool directories. This also further reduced the need for *_binary Bash variables, although it slightly complicated the sequence of commands to obtain preferred build tools due to Windows path handling.

PR description has been updated accordingly.

[kevinAlbs]

LGTM. I like the newly added patching script, and expect that will ease future upgrades of uv.

[eramongodb]

Bumped uv to 0.8.3 to include a fix (astral-sh/uv#14863) for a regression in 0.8.0 concerning --with behavior (astral-sh/uv#14447), although I do not believe it affects any changes in this PR due to no pre-existing project environment being created or used in the scripts being modified (only in the lint task and during a release).

[vector-of-bool]

LGTM with minor comments.

Do we have an open ticket to migrate away from the VS generators? It would greatly simplify a lot of these scripts, but I don't remember whether those tickets were created.

[vector-of-bool]

This particular incantation is repeated several times. Could it be consolidated into a single script file to be dot-sourced?

[eramongodb]

SGTM. Added an install-build-tools.sh which unconditionally obtains cmake and ninja, but in a way which does not require the distro_id variable to detect RHEL, and also in a way that does not import all of MongoDB Toolchain's binaries by adding them to PATH (instead using a simple symlink to "install" only ninja into the uv tool bin directory).

[vector-of-bool]

Is this cmake version pinned?

[eramongodb]

No, it is not. In this case, the make_release.py script is being run manually by the user performing release instructions. However, your question made me realize this should really be made a dependency of the make_release group in pyproject.toml instead. Updated accordingly. cmake is now pinned to 4.0.3 by the lockfile (for the make_release group).

[eramongodb]

Do we have an open ticket to migrate away from the VS generators?

No, I don't think we do. Filed CXX-3326.

[eramongodb]

Bumping the uv version once more before PR merge to 0.8.6 to include the following change:

This release contains hardening measures to address differentials in behavior between uv and Python's built-in ZIP parsers (CVE-2025-54368). [...] Although the practical risk of exploitation is low, we take the hypothetical risk of parser differentials very seriously. Out of an abundance of caution, we have assigned this advisory a CVE identifier and have given it a "moderate" severity suggestion.