Skip to content

fix(pypi): select the lowest available libc version by default #3255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
+92 −12
Draft

fix(pypi): select the lowest available libc version by default #3255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
08d7f77
fix(pypi): select the lowest available libc version by default
aignas Sep 9, 2025
c6ffd3e
document the intention
aignas Sep 15, 2025
ae40e32
test: remove debug flag usage
aignas Sep 15, 2025
2722109
change: always select the lowest
aignas Sep 15, 2025
585ccd8
extra tests
aignas Sep 15, 2025
6b1fc22
update docs
aignas Sep 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,12 @@ END_UNRELEASED_TEMPLATE
length errors due to too long environment variables.
* (bootstrap) {obj}`--bootstrap_impl=script` now supports the `-S` interpreter
setting.
* (pypi) We now use the MVC algorithm to select the right wheel when there are multiple wheels for
the target platform (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user
wants to set the minimum version for the selection algorithm, use the
{attr}`pip.defaults.whl_platform_tags` attribute to configure that. If `musllinux_*_x86_64` is
specified, we will chose the lowest available wheel version. Fixes
[#3250](https://github.com/bazel-contrib/rules_python/issues/3250).

{#v0-0-0-added}
### Added
Expand Down
22 changes: 14 additions & 8 deletions python/private/pypi/extension.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -492,20 +492,22 @@ preference.
Will always include `"any"` even if it is not specified.

The items in this list can contain a single `*` character that is equivalent to matching the
latest available version component in the platform_tag. Note, if the wheel platform tag does not
have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular
character.

We will always select the highest available `platform_tag` version that is compatible with the
target platform.
lowest available version component in the platform_tag. Note, if the wheel platform tag does not
have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular character.

:::{note}
We select a single wheel and the last match will take precedence, if the platform_tag that we
match has a version component (e.g. `android_x_arch`, then the version `x` will be used in the
matching algorithm).

If the matcher you provide has `*`, then we will match a wheel with the highest available target platform, i.e. if `musllinux_1_1_arch` and `musllinux_1_2_arch` are both present, then we will select `musllinux_1_2_arch`.
Otherwise we will select the highest available version that is equal or lower to the specifier, i.e. if `manylinux_2_12` and `manylinux_2_17` wheels are present and the matcher is `manylinux_2_15`, then we will match `manylinux_2_12` but not `manylinux_2_17`.
Normally, the `*` in the matcher means that we will target the lowest platform version that we can
and will give preference to whls built targeting the older versions of the platform. If you
specify the version, then we will use the MVS (Minimal Version Selection) algorithm to select the
compatible wheel. As such, you need to keep in mind how to configure the target platforms to
select a particular wheel of your preference. To sum up:
* To select any wheel, use `*`.
* To exclude versions up to `X.Y` - submit a PR supporting this feature.
* To exclude versions above `X.Y`, provide the full platform tag specifier, e.g. `musllinux_1_2_x86_64`, which will ensure that no wheels with `musllinux_1_3_x86_64` or higher are selected.
:::

:::{note}
Expand All @@ -521,6 +523,10 @@ latest format.
:::{seealso}
See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#platform-tag) for more information.
:::
:::{versionchanged} VERSION_NEXT_FEATURE
The matching of versioned platforms have been switched to MVS (Minimal Version Selection)
algorithm for easier evaluation logic and fewer surprises.
:::
""",
),
} | AUTH_ATTRS
Expand Down
2 changes: 1 addition & 1 deletion python/private/pypi/select_whl.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def _platform_tag_priority(*, tag, values):
# if want_major is defined, then we know that we don't have a `*` in the matcher.
want_version = (int(want_major), int(want_minor)) if want_major else None
if not want_version or version <= want_version:
keys.append((priority, version))
keys.append((priority, (-version[0], -version[1])))

return max(keys) if keys else None

Expand Down
74 changes: 71 additions & 3 deletions tests/pypi/select_whl/select_whl_tests.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,6 @@ def _test_not_select_abi3(env):
whl_abi_tags = ["none"],
python_version = "3.13",
limit = 2,
debug = True,
)
_match(
env,
Expand Down Expand Up @@ -406,9 +405,10 @@ def _test_multiple_musllinux(env):
_match(
env,
got,
# select the one with the highest version that is matching
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
# select the one with the lowest version that is matching because we want to
# increase the compatibility
"pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
)

_tests.append(_test_multiple_musllinux)
Expand All @@ -434,6 +434,74 @@ def _test_multiple_musllinux_exact_params(env):

_tests.append(_test_multiple_musllinux_exact_params)

def _test_multiple_mvs_match(env):
got = _select_whl(
whls = [
"pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
],
whl_platform_tags = ["musllinux_1_3_x86_64"],
whl_abi_tags = ["none"],
python_version = "3.12",
limit = 2,
)
_match(
env,
got,
# select the one with the lowest version
"pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
)

_tests.append(_test_multiple_mvs_match)

def _test_multiple_mvs_match_override_more_specific(env):
got = _select_whl(
whls = [
"pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
],
whl_platform_tags = [
"musllinux_*_x86_64", # default to something
"musllinux_1_3_x86_64", # override the previous
],
whl_abi_tags = ["none"],
python_version = "3.12",
limit = 2,
)
_match(
env,
got,
# Should be the same as without the `*` match
"pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl",
"pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl",
)

_tests.append(_test_multiple_mvs_match_override_more_specific)

def _test_multiple_mvs_match_override_less_specific(env):
got = _select_whl(
whls = [
"pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl",
],
whl_platform_tags = [
"musllinux_1_3_x86_64", # default to 1.3
"musllinux_*_x86_64", # then override to something less specific
],
whl_abi_tags = ["none"],
python_version = "3.12",
limit = 2,
)
_match(
env,
got,
"pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl",
)

_tests.append(_test_multiple_mvs_match_override_less_specific)

def _test_android(env):
got = _select_whl(
whls = [
Expand Down