Invalid annotation of `std::array`

[cielavenir]

It was once reported in #53 , but the issue looks still there...?

• Test resource https://github.com/cielavenir/pybind11_playground/tree/bbddb9318fb2ed8168284c75beacaae9a24ff2c2
• How to test

python3 -m build .
python3 -m pip install dist/pybind11_playground-0.0.0.1-cp39-cp39-linux_x86_64.whl
python3 -m pybind11_stubgen pybind11_playground

mypy stubs/pybind11_playground-stubs/pybind11_playground/__init__.pyi 
stubs/pybind11_playground-stubs/pybind11_playground/__init__.pyi:12: error: "int" expects no type arguments, but 1 given
stubs/pybind11_playground-stubs/pybind11_playground/__init__.pyi:12: error: Invalid type: try using Literal[2] instead?

pyright stubs/pybind11_playground-stubs/pybind11_playground/__init__.pyi
stubs/pybind11_playground-stubs/pybind11_playground/__init__.pyi:12:49 - error: Expected no type arguments for class "int" (reportGeneralTypeIssues)

Expect that the output pyi can be parsed without errors.

[sizmailov]

Pybind generates List[int[2]] annotation for std::array<int, 2>, which is invalid python annotaion as mypy correctly report.

[cielavenir]

@sizmailov So it is pybind11 issue which cannot be fixed by pybind11_stubgen?

[felixvd]

It looks like it: pybind/pybind11#3912

This PR suggests a solution that looks like it could be adapted.

[sizmailov]

@cielavenir, IMO, this should be solved on the pybind side (doc-generation), rather than on stubgen (doc-parsing). Until numpy/numpy#16544 is resolved, the List[int[2]] should be treated as an invalid type.

While it's possible to add List[int[2]] -> List[int] re-mapping on stub-generation stage, I would like to keep stubgen agnostic of mis-generated docstring issues.

If you want to perform such transformations, you can register a custom hook to pre-process docs, e.g:

import pybind11_stubgen
import re

std_array_pattern = re.compile(r"List\[(int|complex|float)[(\d+)]]")


def std_array_fix(match: re.Match):
    return f"List[{match.group(0)}]"


def strip_dimension_from_std_array(docstring: str):
    return std_array_pattern.sub(std_array_fix, docstring)


if __name__ == '__main__':
        
    pybind11_stubgen.function_docstring_preprocessing_hooks.append(
        strip_dimension_from_std_array
    )

    pybind11_stubgen.main()