pylint's type checker ignores type interface files

[jakevdp]

Bug description

First reported at jax-ml/jax#19713, here's a minimal repro:

$ ls
mymodule.py  mymodule.pyi  test.py

$ cat mymodule.py
class Foo:
  pass
setattr(Foo, "__neg__", lambda self: self)

$ cat mymodule.pyi
class Foo:
  def __neg__(self) -> Foo: ...

$ cat test.py
"""Test file for incorrect E1130"""
from mymodule import Foo
result = -Foo()

$ mypy test.py
Success: no issues found in 1 source file

$ pylint test.py
************* Module test
test.py:3:9: E1130: bad operand type for unary -: Foo (invalid-unary-operand-type)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

mypy returns no error because the code is typed correctly: Foo supports __neg__, as declared in the type interface file.

pylint incorrectly errors because it doesn't recognize either the dynamically-defined method or its declaration in the interface file.

Configuration

No response

Command used

see above

Pylint output

see above

Expected behavior

pylint should not error for valid code

Pylint version

pylint 3.0.3
astroid 3.0.3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

OS / Environment

Linux (google Colab)

Additional dependencies

No response

[Pierre-Sassoulas]

Duplicate of #9097, will be fixed by #9241 when we release pylint 3.1.0 with it.

[jakevdp]

Thanks!

[superbobry]

#9097 is not a duplicate of this issue. This issue is not about linting .pyi files in a directory, but rather about using .pyi files during inference.

We have another bug report in google/jax with the same root cause:

# pip install jax==0.4.26 jaxlib==0.4.26
from jax.lib import xla_extension as xe

def main(args):
    jax_fn = xe.pjit("foo", lambda x: x + 1, *args)
    jax_fn(42)  # [not-callable]

pylint concludes the result of xe.pjit is not callable, because the (incorrectly) inferred return type of pjit is None, even though the annotated type in the .pyi is PjitFunction: https://github.com/tensorflow/tensorflow/blob/b468df76ae01a43203a038840efd024946b43e8a/third_party/xla/xla/python/xla_extension/__init__.pyi#L907-L917

Should I file a new issue in astroid?

[DanielNoord]

I'm not sure what the current order is for .pyi files. @jacobtylerwalls might know

[jacobtylerwalls]

Thanks for the comment, but I think it was just that we marked the wrong duplicate. A better duplicate is #9185, which has a fix waiting for review in astroid at pylint-dev/astroid#2375

[superbobry]

Thanks @DanielNoord and @jacobtylerwalls!

This is indeed a more relevant issue/PR, but note that the issue described in my earlier comment is subtly different still. astroid already resolves the import to the .pyi file, since there is no .py file for jax.lib.xla_extension (it's a native module). However, it does not use the type annotations in that .pyi file and concludes that

def pjit(...) -> PjitFunction:
  ...

returns None and not PjitFunction, as annotated.

I assume that the full fix would require revisiting inference logic/heuristics and adjusting them to .pyi files.