Document and improve stubtest

[gvanrossum]

See python/typeshed#1243 (comment)

@JukkaL seems to be volunteering. Also calling out to @jgarst who seems to have worked on this with Jukka during the hack week.

[jgarst]

I'm so sorry for letting this slip for so long. I'll document and apply the suggested edits to the script and try to submit a pull request by tomorrow or the day after.

If you are still open to external contribution, I'd like to contribute here. It is possible I'll just run out of time again, so I'll also try to keep future contributions atomic.

[bluetech]

While working on a PR and grepping for some symbol (namely module_public), I came across this script.

I found that the script no longer works, it uses some broken imports. The following diff gets it to work although I didn't try to figure out what it does or if it works correctly:

Show

diff --git a/scripts/stubtest.py b/scripts/stubtest.py
index 0ec72cef..3fe34c93 100644
--- a/scripts/stubtest.py
+++ b/scripts/stubtest.py
@@ -6,11 +6,12 @@ at runtime.
 
 import importlib
 import sys
-from typing import Dict, Any
+from typing import Dict, Any, List, Iterator
 from collections import defaultdict, namedtuple
 
 from mypy import build
-from mypy.build import default_data_dir, default_lib_path, find_modules_recursive
+from mypy.build import default_data_dir
+from mypy.modulefinder import compute_search_paths, FindModuleCache
 from mypy.errors import CompileError
 from mypy import nodes
 from mypy.options import Options
@@ -54,9 +55,11 @@ Error = namedtuple('Error', (
     'module_type'))
 
 
-def test_stub(name: str):
+def test_stub(options: Options,
+              find_module_cache: FindModuleCache,
+              name: str) -> Iterator[Error]:
     stubs = {
-        mod: stub for mod, stub in build_stubs(name).items()
+        mod: stub for mod, stub in build_stubs(options, find_module_cache, name).items()
         if (mod == name or mod.startswith(name + '.')) and mod not in skip
     }
 
@@ -152,7 +155,7 @@ def verify_typevarexpr(node, module_node):
 
 
 @verify.register(nodes.Decorator)
-def verify_decorator(node, module_noode):
+def verify_decorator(node, module_node):
     if False:
         yield None
 
@@ -162,14 +165,10 @@ def dump_module(name: str) -> Dict[str, Any]:
     return {'type': 'file', 'names': dumpmodule.module_to_json(mod)}
 
 
-def build_stubs(mod):
-    data_dir = default_data_dir(None)
-    options = Options()
-    options.python_version = (3, 6)
-    lib_path = default_lib_path(data_dir,
-                                options.python_version,
-                                custom_typeshed_dir=None)
-    sources = find_modules_recursive(mod, lib_path)
+def build_stubs(options: Options,
+                find_module_cache: FindModuleCache,
+                mod: str) -> Dict[str, nodes.MypyFile]:
+    sources = find_module_cache.find_modules_recursive(mod)
     try:
         res = build.build(sources=sources,
                           options=options)
@@ -184,15 +183,21 @@ def build_stubs(mod):
     return res.files
 
 
-def main(args):
+def main(args: List[str]) -> Iterator[Error]:
     if len(args) == 1:
         print('must provide at least one module to test')
         sys.exit(1)
     else:
         modules = args[1:]
 
+    options = Options()
+    options.python_version = (3, 6)
+    data_dir = default_data_dir()
+    search_path = compute_search_paths([], options, data_dir)
+    find_module_cache = FindModuleCache(search_path)
+
     for module in modules:
-        for error in test_stub(module):
+        for error in test_stub(options, find_module_cache, module):
             yield error
 
 

Given the script was broken since at least 5ac30ea (Tue Mar 13 2018) and no one noticed, perhaps it is better to just remove it? Otherwise, it should at least be checked with mypy, it's pretty good :)

[gvanrossum]

Hey @bluetech, do you want to submit a PR with that diff, so you'll get credit?

I honestly don't know the last time we used this script, and @jgarst seems to have disappeared without finishing docs (somewhat understandable), so if I don't hear from either of you I'll just delete the script...

[bluetech]

Hey @bluetech, do you want to submit a PR with that diff, so you'll get credit?

Not really -- I wouldn't want to submit something I am unsure about.

From my POV, deleting the script would be a good outcome, one less place where module_public is inspected.

[gvanrossum]

I think the script is still useful, it can quickly give you a list of differences between the stub for a module and the actual module. Do you want to submit a PR that includes it in the mypy checking CI?

…

On Tue, Jul 30, 2019 at 12:25 PM Ran Benita ***@***.***> wrote: Hey @bluetech <https://github.com/bluetech>, do you want to submit a PR with that diff, so you'll get credit? Not really -- I wouldn't want to submit something I am unsure about. From my POV, deleting the script would be a good outcome, one less place where module_public is inspected. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3329?email_source=notifications&email_token=AAWCWMQS7HUXXS4K3D7326LQCCIRJA5CNFSM4DKITFA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3FA7SA#issuecomment-516558792>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWCWMVVUNHNR3NTX74GVJ3QCCIRJANCNFSM4DKITFAQ> .

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>