Python 2 removal part 1: tests (C++ code is intentionally ~untouched) (#3688)

* `#error BYE_BYE_GOLDEN_SNAKE`

* Removing everything related to 2.7 from ci.yml

* Commenting-out Centos7

* Removing `PYTHON: 27` from .appveyor.yml

* "PY2" removal, mainly from tests. C++ code is not touched.

* Systematic removal of `u` prefix from `u"..."` and `u'...'` literals. Collateral cleanup of a couple minor other things.

* Cleaning up around case-insensitive hits for `[^a-z]py.*2` in tests/.

* Removing obsolete Python 2 mention in compiling.rst

* Proper `#error` for Python 2.

* Using PY_VERSION_HEX to guard `#error "PYTHON 2 IS NO LONGER SUPPORTED.`

* chore: bump pre-commit

* style: run pre-commit for pyupgrade 3+

* tests: use sys.version_info, not PY

* chore: more Python 2 removal

* Uncommenting Centos7 block (PR #3691 showed that it is working again).

* Update pre-commit hooks

* Fix pre-commit hook

* refactor: remove Python 2 from CMake

* refactor: remove Python 2 from setup code

* refactor: simplify, better static typing

* feat: fail with nice messages

* refactor: drop Python 2 C++ code

* docs: cleanup for Python 3

* revert: intree

revert: intree

* docs: minor touchup to py2 statement

Co-authored-by: Henry Schreiner <[email protected]>
Co-authored-by: Aaron Gokaslan <[email protected]>

Author: 3 people

.appveyor.yml

@@ -11,8 +11,6 @@ environment:
   matrix:
   - PYTHON: 36
     CONFIG: Debug
-  - PYTHON: 27
-    CONFIG: Debug
 install:
 - ps: |
     $env:CMAKE_GENERATOR = "Visual Studio 14 2015"

.github/workflows/ci.yml

@@ -25,7 +25,6 @@ jobs:
       matrix:
         runs-on: [ubuntu-latest, windows-2022, macos-latest]
         python:
-        - '2.7'
         - '3.5'
         - '3.6'
         - '3.9'
@@ -49,13 +48,9 @@ jobs:
             python: '3.6'
             args: >
               -DPYBIND11_FINDPYTHON=ON
-          - runs-on: macos-latest
-            python: 'pypy-2.7'
           # Inject a couple Windows 2019 runs
           - runs-on: windows-2019
             python: '3.9'
-          - runs-on: windows-2019
-            python: '2.7'
 
     name: "🐍 ${{ matrix.python }} • ${{ matrix.runs-on }} • x64 ${{ matrix.args }}"
     runs-on: ${{ matrix.runs-on }}
@@ -168,22 +163,6 @@ jobs:
     - name: Interface test
       run: cmake --build build2 --target test_cmake_build
 
-    # Eventually Microsoft might have an action for setting up
-    # MSVC, but for now, this action works:
-    - name: Prepare compiler environment for Windows 🐍 2.7
-      if: matrix.python == 2.7 && runner.os == 'Windows'
-      uses: ilammy/[email protected]
-      with:
-        arch: x64
-
-    # This makes two environment variables available in the following step(s)
-    - name: Set Windows 🐍 2.7 environment variables
-      if: matrix.python == 2.7 && runner.os == 'Windows'
-      shell: bash
-      run: |
-        echo "DISTUTILS_USE_SDK=1" >> $GITHUB_ENV
-        echo "MSSdk=1" >> $GITHUB_ENV
-
     # This makes sure the setup_helpers module can build packages using
     # setuptools
     - name: Setuptools helpers test
@@ -784,7 +763,6 @@ jobs:
       fail-fast: false
       matrix:
         python:
-          - 2.7
           - 3.6
           - 3.7
           # todo: check/cpptest does not support 3.8+ yet
@@ -832,17 +810,12 @@ jobs:
       fail-fast: false
       matrix:
         python:
-          - 2.7
           - 3.5
           - 3.7
         std:
           - 14
 
         include:
-          - python: 2.7
-            std: 17
-            args: >
-              -DCMAKE_CXX_FLAGS="/permissive- /EHsc /GR"
           - python: 3.7
             std: 17
             args: >

.github/workflows/pip.yml

@@ -17,19 +17,19 @@ env:
 
 jobs:
   # This builds the sdists and wheels and makes sure the files are exactly as
-  # expected. Using Windows and Python 2.7, since that is often the most
+  # expected. Using Windows and Python 3.6, since that is often the most
   # challenging matrix element.
   test-packaging:
-    name: 🐍 2.7 • 📦 tests • windows-latest
+    name: 🐍 3.6 • 📦 tests • windows-latest
     runs-on: windows-latest
 
     steps:
     - uses: actions/checkout@v2
 
-    - name: Setup 🐍 2.7
+    - name: Setup 🐍 3.6
       uses: actions/setup-python@v2
       with:
-        python-version: 2.7
+        python-version: 3.6
 
     - name: Prepare env
       run: |

.pre-commit-config.yaml

@@ -29,35 +29,37 @@ repos:
   - id: mixed-line-ending
   - id: requirements-txt-fixer
   - id: trailing-whitespace
-  - id: fix-encoding-pragma
-    exclude: ^noxfile.py$
 
+# Upgrade old Python syntax
 - repo: https://github.com/asottile/pyupgrade
   rev: v2.31.0
   hooks:
   - id: pyupgrade
+    args: [--py3-plus]
 
+# Nicely sort includes
 - repo: https://github.com/PyCQA/isort
   rev: 5.10.1
   hooks:
   - id: isort
 
 # Black, the code formatter, natively supports pre-commit
 - repo: https://github.com/psf/black
-  rev: 21.12b0 # Keep in sync with blacken-docs
+  rev: 22.1.0 # Keep in sync with blacken-docs
   hooks:
   - id: black
 
+# Also code format the docs
 - repo: https://github.com/asottile/blacken-docs
-  rev: v1.12.0
+  rev: v1.12.1
   hooks:
   - id: blacken-docs
     additional_dependencies:
-    - black==21.12b0 # keep in sync with black hook
+    - black==22.1.0 # keep in sync with black hook
 
 # Changes tabs to spaces
 - repo: https://github.com/Lucas-C/pre-commit-hooks
-  rev: v1.1.10
+  rev: v1.1.11
   hooks:
   - id: remove-tabs
 
@@ -67,12 +69,15 @@ repos:
   hooks:
   - id: pycln
 
+# Checking for common mistakes
 - repo: https://github.com/pre-commit/pygrep-hooks
   rev: v1.9.0
   hooks:
   - id: python-check-blanket-noqa
   - id: python-check-blanket-type-ignore
   - id: python-no-log-warn
+  # Python 3.6
+  # - id: python-use-type-annotations
   - id: rst-backticks
   - id: rst-directive-colons
   - id: rst-inline-touching-normal
@@ -87,6 +92,7 @@ repos:
       - pep8-naming
     exclude: ^(docs/.*|tools/.*)$
 
+# Automatically remove noqa that are not used
 - repo: https://github.com/asottile/yesqa
   rev: v1.3.0
   hooks:
@@ -107,9 +113,9 @@ repos:
   rev: v0.931
   hooks:
   - id: mypy
-    # Running per-file misbehaves a bit, so just run on all files, it's fast
-    pass_filenames: false
-    additional_dependencies: [typed_ast]
+    args: [--show-error-codes]
+    exclude: ^(tests|docs)/
+    additional_dependencies: [nox, rich]
 
 # Checks the manifest for missing files (native support)
 - repo: https://github.com/mgedmin/check-manifest
@@ -120,27 +126,30 @@ repos:
     stages: [manual]
     additional_dependencies: [cmake, ninja]
 
+# Check for spelling
 - repo: https://github.com/codespell-project/codespell
   rev: v2.1.0
   hooks:
   - id: codespell
     exclude: ".supp$"
     args: ["-L", "nd,ot,thist"]
 
+# Check for common shell mistakes
 - repo: https://github.com/shellcheck-py/shellcheck-py
-  rev: v0.8.0.3
+  rev: v0.8.0.4
   hooks:
   - id: shellcheck
 
-# The original pybind11 checks for a few C++ style items
+# Disallow some common capitalization mistakes
 - repo: local
   hooks:
   - id: disallow-caps
     name: Disallow improper capitalization
     language: pygrep
     entry: PyBind|Numpy|Cmake|CCache|PyTest
-    exclude: .pre-commit-config.yaml
+    exclude: ^\.pre-commit-config.yaml$
 
+# Clang format the codebase automatically
 - repo: https://github.com/pre-commit/mirrors-clang-format
   rev: "v13.0.0"
   hooks:

MANIFEST.in

@@ -1,6 +1,5 @@
 recursive-include pybind11/include/pybind11 *.h
 recursive-include pybind11 *.py
 recursive-include pybind11 py.typed
-recursive-include pybind11 *.pyi
 include pybind11/share/cmake/pybind11/*.cmake
 include LICENSE README.rst pyproject.toml setup.py setup.cfg

README.rst

@@ -34,7 +34,7 @@ dependency.
 Think of this library as a tiny self-contained version of Boost.Python
 with everything stripped away that isn’t relevant for binding
 generation. Without comments, the core header files only require ~4K
-lines of code and depend on Python (2.7 or 3.5+, or PyPy) and the C++
+lines of code and depend on Python (3.5+, or PyPy) and the C++
 standard library. This compact implementation was possible thanks to
 some of the new C++11 language features (specifically: tuples, lambda
 functions and variadic templates). Since its creation, this library has
@@ -78,8 +78,8 @@ Goodies
 In addition to the core functionality, pybind11 provides some extra
 goodies:
 
-- Python 2.7, 3.5+, and PyPy/PyPy3 7.3 are supported with an
-  implementation-agnostic interface.
+- Python 3.5+, and PyPy3 7.3 are supported with an implementation-agnostic
+  interface (pybind11 2.9 was the last version to support Python 2).
 
 - It is possible to bind C++11 lambda functions with captured
   variables. The lambda capture data is stored inside the resulting

docs/Doxyfile

@@ -18,5 +18,4 @@ ALIASES               += "endrst=\endverbatim"
 QUIET                  = YES
 WARNINGS               = YES
 WARN_IF_UNDOCUMENTED   = NO
-PREDEFINED             = PY_MAJOR_VERSION=3 \
-                         PYBIND11_NOINLINE
+PREDEFINED             = PYBIND11_NOINLINE

docs/advanced/cast/strings.rst

@@ -1,14 +1,6 @@
 Strings, bytes and Unicode conversions
 ######################################
 
-.. note::
-
-    This section discusses string handling in terms of Python 3 strings. For
-    Python 2.7, replace all occurrences of ``str`` with ``unicode`` and
-    ``bytes`` with ``str``.  Python 2.7 users may find it best to use ``from
-    __future__ import unicode_literals`` to avoid unintentionally using ``str``
-    instead of ``unicode``.
-
 Passing Python strings to C++
 =============================
 
@@ -58,9 +50,9 @@ Passing bytes to C++
 --------------------
 
 A Python ``bytes`` object will be passed to C++ functions that accept
-``std::string`` or ``char*`` *without* conversion.  On Python 3, in order to
-make a function *only* accept ``bytes`` (and not ``str``), declare it as taking
-a ``py::bytes`` argument.
+``std::string`` or ``char*`` *without* conversion.  In order to make a function
+*only* accept ``bytes`` (and not ``str``), declare it as taking a ``py::bytes``
+argument.
 
 
 Returning C++ strings to Python
@@ -204,11 +196,6 @@ decoded to Python ``str``.
         }
     );
 
-.. warning::
-
-    Wide character strings may not work as described on Python 2.7 or Python
-    3.3 compiled with ``--enable-unicode=ucs2``.
-
 Strings in multibyte encodings such as Shift-JIS must transcoded to a
 UTF-8/16/32 before being returned to Python.
 

docs/advanced/classes.rst

@@ -133,14 +133,14 @@ a virtual method call.
     >>> from example import *
     >>> d = Dog()
     >>> call_go(d)
-    u'woof! woof! woof! '
+    'woof! woof! woof! '
     >>> class Cat(Animal):
     ...     def go(self, n_times):
     ...         return "meow! " * n_times
     ...
     >>> c = Cat()
     >>> call_go(c)
-    u'meow! meow! meow! '
+    'meow! meow! meow! '
 
 If you are defining a custom constructor in a derived Python class, you *must*
 ensure that you explicitly call the bound C++ constructor using ``__init__``,
@@ -813,26 +813,21 @@ An instance can now be pickled as follows:
 
 .. code-block:: python
 
-    try:
-        import cPickle as pickle  # Use cPickle on Python 2.7
-    except ImportError:
-        import pickle
+    import pickle
 
     p = Pickleable("test_value")
     p.setExtra(15)
-    data = pickle.dumps(p, 2)
+    data = pickle.dumps(p)
 
 
 .. note::
-    Note that only the cPickle module is supported on Python 2.7.
-
-    The second argument to ``dumps`` is also crucial: it selects the pickle
-    protocol version 2, since the older version 1 is not supported. Newer
-    versions are also fine—for instance, specify ``-1`` to always use the
-    latest available version. Beware: failure to follow these instructions
-    will cause important pybind11 memory allocation routines to be skipped
-    during unpickling, which will likely lead to memory corruption and/or
-    segmentation faults.
+    If given, the second argument to ``dumps`` must be 2 or larger - 0 and 1 are
+    not supported. Newer versions are also fine; for instance, specify ``-1`` to
+    always use the latest available version. Beware: failure to follow these
+    instructions will cause important pybind11 memory allocation routines to be
+    skipped during unpickling, which will likely lead to memory corruption
+    and/or segmentation faults. Python defaults to version 3 (Python 3-3.7) and
+    version 4 for Python 3.8+.
 
 .. seealso::
 
@@ -849,11 +844,9 @@ Python normally uses references in assignments. Sometimes a real copy is needed
 to prevent changing all copies. The ``copy`` module [#f5]_ provides these
 capabilities.
 
-On Python 3, a class with pickle support is automatically also (deep)copy
+A class with pickle support is automatically also (deep)copy
 compatible. However, performance can be improved by adding custom
-``__copy__`` and ``__deepcopy__`` methods. With Python 2.7, these custom methods
-are mandatory for (deep)copy compatibility, because pybind11 only supports
-cPickle.
+``__copy__`` and ``__deepcopy__`` methods.
 
 For simple classes (deep)copy can be enabled by using the copy constructor,
 which should look as follows:

docs/advanced/exceptions.rst

@@ -328,8 +328,8 @@ an invalid state.
 Chaining exceptions ('raise from')
 ==================================
 
-In Python 3.3 a mechanism for indicating that exceptions were caused by other
-exceptions was introduced:
+Python has a mechanism for indicating that exceptions were caused by other
+exceptions:
 
 .. code-block:: py
 
@@ -340,7 +340,7 @@ exceptions was introduced:
 
 To do a similar thing in pybind11, you can use the ``py::raise_from`` function. It
 sets the current python error indicator, so to continue propagating the exception
-you should ``throw py::error_already_set()`` (Python 3 only).
+you should ``throw py::error_already_set()``.
 
 .. code-block:: cpp