stubtest: error if a dunder method is missing from a stub #12203

AlexWaygood · 2022-02-17T12:27:54Z

Description

This PR alters stubtest so that it raises an error if a dunder method is missing from a stub. Currently, stubtest does raise an error for a few missing dunders (__call__, __class_getitem__, etc.). However, for the vast majority of them, it does not raise an error.

I have been using this patch to provide a number of fixes to typeshed over the past week or so:

With these typeshed fixes merged, here are the new errors stubtest reports from checking typeshed, with this patch applied:

+ _collections_abc.Mapping.__eq__ is inconsistent, stub argument "__o" should be positional or keyword (remove leading double underscore)
+ _collections_abc.Mapping.__reversed__ is not present in stub
+ _collections_abc.Set.__eq__ is inconsistent, stub argument "__o" should be positional or keyword (remove leading double underscore)
+ _collections_abc.Set.__rand__ is not present in stub
+ _collections_abc.Set.__ror__ is not present in stub
+ _collections_abc.Set.__rsub__ is not present in stub
+ _collections_abc.Set.__rxor__ is not present in stub
+ _collections_abc.dict_items.__and__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__contains__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__key")
+ _collections_abc.dict_items.__ge__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__gt__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__le__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__lt__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__or__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__rand__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__ror__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__rsub__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__rxor__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__sub__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_items.__xor__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__and__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__contains__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__key")
+ _collections_abc.dict_keys.__ge__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__gt__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__le__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__lt__ is inconsistent, stub argument "s" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__or__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__rand__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__ror__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__rsub__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__rxor__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__sub__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _collections_abc.dict_keys.__xor__ is inconsistent, stub argument "o" should be positional-only (rename with a leading double underscore, i.e. "__value")
+ _weakref.ProxyType.__bytes__ is not present in stub
+ _weakref.ProxyType.__reversed__ is not present in stub
+ builtins.property.__set_name__ is not present in stub
+ builtins.super.__self__ is not present in stub
+ builtins.super.__self_class__ is not present in stub
+ builtins.super.__thisclass__ is not present in stub
+ collections.ChainMap.__bool__ is not present in stub
+ collections.Counter.__missing__ is not present in stub
+ dbm.dumb._Database.__contains__ is inconsistent, stub argument "__o" should be positional or keyword (remove leading double underscore)
+ enum.EnumMeta.__prepare__ is inconsistent, stub argument "__name" should be positional or keyword (remove leading double underscore)
+ enum.EnumMeta.__prepare__ is inconsistent, stub argument "__bases" should be positional or keyword (remove leading double underscore)
+ functools.partial.__vectorcalloffset__ is not present in stub
+ http.cookies.BaseCookie.__str__ is inconsistent, stub does not have argument "attrs"
+ http.cookies.BaseCookie.__str__ is inconsistent, stub does not have argument "header"
+ http.cookies.BaseCookie.__str__ is inconsistent, stub does not have argument "sep"
+ http.cookies.Morsel.__str__ is inconsistent, stub does not have argument "attrs"
+ http.cookies.Morsel.__str__ is inconsistent, stub does not have argument "header"
+ lib2to3.pytree.Leaf.__unicode__ is not present in stub
+ lib2to3.pytree.Node.__unicode__ is not present in stub
+ platform.uname_result.__getitem__ is inconsistent, stub argument "__x" should be positional or keyword (remove leading double underscore)
+ pstats.FunctionProfile.__eq__ is inconsistent, stub argument "__o" should be positional or keyword (remove leading double underscore)
+ pstats.StatsProfile.__eq__ is inconsistent, stub argument "__o" should be positional or keyword (remove leading double underscore)
+ types.FunctionType.__builtins__ is not present in stub
+ types.LambdaType.__builtins__ is not present in stub
+ uuid.UUID.__setattr__ is inconsistent, stub argument "__name" should be positional or keyword (remove leading double underscore)
+ uuid.UUID.__setattr__ is inconsistent, stub argument "__value" should be positional or keyword (remove leading double underscore)
+ weakref.ProxyType.__bytes__ is not present in stub
+ weakref.ProxyType.__reversed__ is not present in stub
+ xml.dom.minidom.Node.__bool__ is not present in stub

Notes on the new errors reported

We have a few new complaints about discrepancies between typing and collections.abc. These just need to be allowlisted imo, unless and until we find a good general solution to that problem (refs Decouple collections.abc types from typing aliases typeshed#6257)
There are a lot of complaints about dict_keys, dict_values and dict_items having positional-only differences with typing.KeysView, typing.ValuesView and typing.ItemsView. The way to resolve that would be essentially duplicate every method from typing.{Keys, Values, Items}View into those classes (but with pos-only args instead of pos-or-kw args). I don't really have the stomach to do that. These should probably just be allowlisted.
weakref.ProxyType, dbm.dum._Database, lib2to3.pytree.Leaf: I don't really know anything about these classes.
enum.EnumMeta.__prepare__: this will be resolved by Add EnumMeta.__prepare__ typeshed#7243
super has some really weird dunders.
property.__set_name__: this is a false positive -- it doesn't exist in real life, just needs to be allowlisted.
http.cookies has some really weird __str__ methods.
functools.partial.__vectorcalloffset__: no idea what this does or is for.
platform.uname_result is a mess in CPython.
pstats hits: these relate to a mypy bug in dataclasses __eq__ method generation (Mypy incorrectly infers that parameters are positional-only for __eq__ methods autogenerated with @dataclass #12186).
collections.Counter.__missing__: this always raises an exception, but per Why is collections.ChainMap.__missing__ defined in the stub? typeshed#7188, we could maybe add it to the stub.
There's a few missing __bool__ overrides stubtest now complains about: we could add these to the stub, though they don't seem particularly useful, since they don't alter the default behaviour of bool(x) at all. (But I don't think __bool__ should be added to the list of ignored dunders: it was useful for this patch to flag the missing __bool__ method fixed in Add EnumMeta.__bool__ typeshed#7206, as that is an override that changes behaviour from the default.)
uuid.UUID.__setattr__: this always raises an exception, I think it should just be allowlisted.

Test Plan

I've altered the existing tests so they continue to pass, and added two new test cases.

cc. @hauntsaninja: here is the promised follow-up PR!

AlexWaygood · 2022-02-17T15:21:59Z

mypy/stubtest.py

+IGNORED_DUNDERS = frozenset({
+    # Very special attributes
+    "__weakref__",
+    "__slots__",
+    "__dict__",
+    "__text_signature__",
+    "__match_args__",
+    # Pickle methods
+    "__setstate__",
+    "__getstate__",
+    "__getnewargs__",
+    "__getinitargs__",
+    "__reduce_ex__",
+    "__reduce__",
+    # typing implementation details
+    "__parameters__",
+    "__origin__",
+    "__args__",
+    "__orig_bases__",
+    "__mro_entries__",
+    "__forward_is_class__",
+    "__forward_module__",
+    "__final__",
+    # isinstance/issubclass hooks that type-checkers don't usually care about
+    "__instancecheck__",
+    "__subclasshook__",
+    "__subclasscheck__",
+    # Dataclasses implementation details
+    "__dataclass_fields__",
+    "__dataclass_params__",
+    # ctypes weirdness
+    "__ctype_be__",
+    "__ctype_le__",
+    "__ctypes_from_outparam__",
+    # Two float methods only used internally for CPython test suite, not for public use
+    "__set_format__",
+    "__getformat__",
+    # These two are basically useless for type checkers
+    "__hash__",
+    "__getattr__",
+    # For some reason, mypy doesn't infer classes with metaclass=ABCMeta inherit this attribute
+    "__abstractmethods__",
+    "__doc__",  # Can only ever be str | None, who cares?
+    "__del__",  # Only ever called when an object is being deleted, who cares?
+    "__new_member__",  # If an enum defines __new__, the method is renamed as __new_member__
+})


This is a fairly arbitrary ignorelist, based on the hits that this patch initially came up with, combined with my judgement as to which were worth fixing and which weren't. I'm happy to add or remove items!

mypy/stubtest.py

…tubtest

mypy/stubtest.py

hauntsaninja

This is great, thanks for doing this!

github-actions · 2022-02-19T23:41:41Z

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

AlexWaygood · 2022-02-19T23:52:47Z

Thanks! :D

Basically a follow up to #12203 New errors in typeshed from this: ``` _decimal.__libmpdec_version__ is not present in stub _heapq.__about__ is not present in stub builtins.__build_class__ is not present in stub cgitb.__UNDEF__ is not present in stub decimal.__libmpdec_version__ is not present in stub sys.__unraisablehook__ is not present in stub ``` Some general housekeeping, moving things around, renaming things, adding some comments.

This isn't actually a reversion. This logic was asymmetrical for reasons lost to time. Although I suspect it was this way because it introduced only a few errors on typeshed. What changed was that as a result of python#12203 we actually started checking a lot more dunder methods. Previously, we only checked dunders if either: a) it was a special dunder, like __init__ or __call, or b) the dunder was defined on the actual stub class itself In particular, we started checking every dunder redefined at runtime that the stub just inherited from object. A possible intermediate option would be to not check positional-only arguments in the case where a stub inherits the definition. I'll experiment with that once typeshed CI is green.

This isn't actually a reversion. This logic was asymmetrical for reasons lost to time. Although I suspect it was this way because the delta only a few errors on typeshed. What changed was that as a result of #12203 we actually started checking a lot more dunder methods. Previously, we only checked dunders if either: a) it was a special dunder, like `__init__` or `__call__`, or b) the dunder was defined on the actual stub class itself In particular, we started checking every dunder redefined at runtime that the stub just inherited from object. A possible intermediate option would be to not check positional-only arguments in the case where a stub inherits the definition. I'll experiment with that once typeshed CI is green. Co-authored-by: hauntsaninja <>

AlexWaygood added 3 commits February 17, 2022 11:57

stubtest: error if a dunder method is missing from a stub

62b5875

flake8

8324e2f

py36 fixes (ew)

ce0cc14

This comment has been minimized.

Sign in to view

97littleleaf11 requested a review from hauntsaninja February 17, 2022 14:22

Merge remote-tracking branch 'origin/master' into stubtest

8e49d6b

This comment was marked as outdated.

Sign in to view

AlexWaygood added 2 commits February 17, 2022 15:09

Make tests actually test what I want them to test

4425a3b

Merge branch 'python:master' into stubtest

6195e20

AlexWaygood commented Feb 17, 2022

View reviewed changes

JelleZijlstra reviewed Feb 17, 2022

View reviewed changes

mypy/stubtest.py Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

AlexWaygood added 2 commits February 18, 2022 14:52

Improve py36 workaround, add comment re __match_args__

ba25ed0

Merge branch 'stubtest' of https://github.com/AlexWaygood/mypy into s…

52097ba

…tubtest

This comment has been minimized.

Sign in to view

Merge branch 'python:master' into stubtest

e7080a9

This comment has been minimized.

Sign in to view

hauntsaninja reviewed Feb 19, 2022

View reviewed changes

mypy/stubtest.py Outdated Show resolved Hide resolved

mypy/stubtest.py Outdated Show resolved Hide resolved

mypy/stubtest.py Show resolved Hide resolved

mypy/stubtest.py Outdated Show resolved Hide resolved

mypy/stubtest.py Show resolved Hide resolved

hauntsaninja reviewed Feb 19, 2022

View reviewed changes

AlexWaygood added 4 commits February 19, 2022 22:40

Remove 5 methods from the ignorelist

749b2c3

Get rid of 3rd arg in getattr call

cfb4c3a

Move logic from verify_none into verify_typeinfo

a71b1b3

Death to the is_dunder_slot_wrapper helper function

1720eaa

AlexWaygood requested a review from hauntsaninja February 19, 2022 23:43

hauntsaninja merged commit 9af578d into python:master Feb 19, 2022

AlexWaygood deleted the stubtest branch February 19, 2022 23:52

hauntsaninja mentioned this pull request Feb 20, 2022

stubtest: error if module level dunder is missing, housekeeping #12217

Merged

This was referenced Feb 22, 2022

Stubtest failed on Tue Feb 22 2022 python/typeshed#7344

Closed

uuid: mark setattr python/typeshed#7347

Closed

AlexWaygood mentioned this pull request Feb 26, 2022

Enum values on enum class can't be reassigned or deleted python/typeshed#7385

Closed

This was referenced Mar 5, 2022

stubtest: ignore more dunder pos-only errors #12294

Merged

Stubtest failed on Sat Mar 05 2022 python/typeshed#7441

Closed

AlexWaygood mentioned this pull request Mar 21, 2022

Fix inheritance false positives with dataclasses/attrs #12411

Merged

AlexWaygood mentioned this pull request Sep 13, 2022

refactor: update parsimonious to 0.10.0 python/typeshed#8730

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

stubtest: error if a dunder method is missing from a stub #12203

stubtest: error if a dunder method is missing from a stub #12203

Uh oh!

AlexWaygood commented Feb 17, 2022 •

edited

Loading

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

AlexWaygood Feb 17, 2022 •

edited

Loading

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hauntsaninja left a comment

Uh oh!

github-actions bot commented Feb 19, 2022

Uh oh!

AlexWaygood commented Feb 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

stubtest: error if a dunder method is missing from a stub #12203

stubtest: error if a dunder method is missing from a stub #12203

Uh oh!

Conversation

AlexWaygood commented Feb 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes on the new errors reported

Test Plan

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

AlexWaygood Feb 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hauntsaninja left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 19, 2022

Uh oh!

AlexWaygood commented Feb 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexWaygood commented Feb 17, 2022 •

edited

Loading

AlexWaygood Feb 17, 2022 •

edited

Loading