We need a policy for undocumented attributes

[gvanrossum]

In the past week alone I've seen several issues where well-meaning contributors wanted to add undocumented attributes to typeshed:

• Add _taskqueue to MP Pool #1884
• Extend Py3 Queue type hints #1885
• Add stub for subprocess.list2cmdline function #1898
• Python 3 HTTPConnection stubs missing class variables #1901

I think we should have a standard policy on these. I know in the past we've sometimes added such attributes, with the argument that there's existing working code that uses them. But AFAIK there's no other tool that is capable of flagging the use of undocumented attributes, so I'm kind of leaning towards seeing this as a valuable service mypy (or rather, typeshed) provides, rather than a deficiency in the stubs. For example, if we were to add these attributes to the stubs, IDEs like PyCharm would start suggesting auto-completion for undocumented attributes, which I definitely think is a bad idea. If there's something that's commonly used, it should just be included in the docs first -- the process for that is clear (file an issue on bugs.python.org).

[ambv]

This issue is why I think we need more than just a type checker. Warnings against deprecations, use of dangerous APIs, invalid combinations of arguments, etc., is something that mypy could do, even though this functionality is not related to typing. My pet example is warning against instantiating coroutines without await or an assignment.

With all the above said, typeshed documents types and mypy checks types. We are generally working pretty hard to not raise false positives because they undermine trust in the tool. "Hey, mypy is saying there's no function like that but I can see it in dir(), it's documented with help() and works at runtime!"

So, my position is this:

• we should let any and all pull requests in that provide valid type information, including for private/undocumented/deprecated functions/classes;
• we need to have a talk around extending mypy in ways that helps with more than just checking types. mypy could be the most powerful static analysis tool for Python ever, running circles around linters. This discussion is a bit out of scope for typeshed, but...

If we decide that mypy should indeed allow for additional warnings that aren't type errors, then extending the .pyi syntax with a standard way to mark functions/classes that should not be used, would be nice. It could be as simple as @do_not_use("implementation detail, can change without warning in a future Python version") and/or @deprecated("will be removed in Python 3.8"). But I don't want to bikeshed here.

[gvanrossum]

I talked briefly with @JukkaL about this and he agrees with you. I propose that while we ponder how mypy can be improved to warn about the use of undocumented attributes, we add a standardized comment # undocumented to those attributes (notes: no # type: -- and this can be appended to a regular # type: XXX comments.

[gvanrossum]

So can we close this now?

Actually we should do something similar for deprecated APIs perhaps?

[srittau]

# undocumented was documented (no pun intended) in #1938.