[BUG] python 3.12 does not have distutils (setuptools imports distutils)

[matteius]

setuptools version

68.0.0

Python version

3.12

OS

Ubunut Linux, WIndows, Mac OS

Additional environment information

I maintain pipenv and am trying to get enable python 3.12 support.

Description

Setuptools imports distutils: https://github.com/pypa/setuptools/blob/main/setuptools/__init__.py#L9

Its not an issue on my local Windows 3.12 because I can import distutils, but on the Github CI it fails: https://github.com/pypa/pipenv/actions/runs/5773881521/job/15650307431

What is the path forward given that python 3.12 drops distutils?

Expected behavior

I would expect setuptools would stop using disutils, but maybe there is some other piece of information I am missing.

How to Reproduce

https://github.com/pypa/pipenv/actions/runs/5773881521/job/15650307431

Output

Traceback (most recent call last):
  File "/home/runner/work/pipenv/pipenv/pipenv/resolver.py", line 699, in <module>
    main()
  File "/home/runner/work/pipenv/pipenv/pipenv/resolver.py", line 685, in main
    _main(
  File "/home/runner/work/pipenv/pipenv/pipenv/resolver.py", line 665, in _main
    resolve_packages(
  File "/home/runner/work/pipenv/pipenv/pipenv/resolver.py", line 632, in resolve_packages
    results, resolver = resolve(
                        ^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/resolver.py", line 612, in resolve
    return resolve_deps(
           ^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/utils/resolver.py", line 842, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
                                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/utils/resolver.py", line 617, in actually_resolve_deps
    resolver.resolve()
  File "/home/runner/work/pipenv/pipenv/pipenv/utils/resolver.py", line 439, in resolve
    results = resolver.resolve(self.constraints, check_supported_wheels=False)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
                ^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 157, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/operations/prepare.py", line 653, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pipenv.patched.pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/home/runner/work/pipenv/pipenv/pipenv/patched/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.0-beta.4/x64/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1216, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1293, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1266, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1237, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 841, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 994, in exec_module
  File "<frozen importlib._bootstrap>", line 400, in _call_with_frames_removed
  File "/tmp/pip-build-env-3jy_4xzk/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 5, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

[matteius]

Actually it may be that the private pypi that some of the test run against are pulling an older setuptools -- I just added the latest 68.0.0 and am re-running the tests. 🤞

[matteius]

Yeah, that seems to have been my issue.

[madig]

FWIW, I ran into the same problem and had to remove the SETUPTOOLS_USE_DISTUTILS=stdlib env var I set years ago and forgot about.

[sgillies]

@madig thanks for writing that! I have been having the same problem and was stumped.

[bashtage]

FWIW, I ran into the same problem and had to remove the SETUPTOOLS_USE_DISTUTILS=stdlib env var I set years ago and forgot about.

@madig I had the exact same issue -- thank for posting the fix.