Python version per platform support for extension authors

[echarles]

A first server extension being developed is JupyterLab which today supports and CI tests on Linux and Windows for python 3.5, 3.6, 3.7, 3.8. The specific JupyterLab case depends also on nbclassic.

I list here after the restrictions imposed today in the various setup.py

• jupyterlab: linux+win32(py >= 3.5)
• jupyter_server: linux(py >= 3.5) win32(py >= 3.7)
• nbclassic: linux+win32(py >= 3.6)

I list then the CI tests being run

• Linux py-[3.5, 3.6, 3.7, 3.8]
• Max py-[3.5, 3.6, 3.7, 3.8]
• Windows py-[3.7, 3.8]

At runtime, some restrictions also apply in jupyter_server with AsyncMappingKernelManager: py >= 3.6, see https://github.com/jupyter/jupyter_server/blob/4f2d91d1754a30c9e46b48638c5a67907729fcb6/jupyter_server/serverapp.py#L1262-L1269

Regarding Windows with py3.8 we have a fix https://github.com/jupyter/jupyter_server/blob/4f2d91d1754a30c9e46b48638c5a67907729fcb6/jupyter_server/serverapp.py#L1735-L1760 and CI is green for Windows py-3.8 https://github.com/jupyter/jupyter_server/runs/631446097?check_suite_focus=true and I have not found test exclusion for that case so I guess server is fine on Windows py-3.8

Assuming JupyterLab 3.0 would drop python 3.5 support, the remaining actions would be:

• We need to review our Windows support for 3.6
• For nbclassic https://github.com/Zsailer/nbclassic/issues/11 is optional

@Zsailer @blink1073 @afshin @vidartf @kevin-bates @davidbrochart

[blink1073]

cc @jasongrout

[blink1073]

Given that Python 3.5 reaches end of life on 2020-09-13, and we will be supporting JupyterLab 2.0 past that date, I say we drop Python 3.5 for JupyterLab 3.0.

[blink1073]

The lack of Python 3.6 support on Windows is an issue though.

[blink1073]

Is it just that AsyncMappingKernelManager does not work on Windows for Python 3.6? If we can have it degrade gracefully, and just skip a test or two for that scenario, I think we are fine.

[echarles]

I have not followed closely enough the work around AsyncMappingKernelManager, but I feel this is the case. @kevin-bates or @davidbrochart can confirm this and hopefully open a PR to skip the needed tests that fail the AsyncMappingKernelManager on Windows with Python 3.6.

[davidbrochart]

Is it just that AsyncMappingKernelManager does not work on Windows for Python 3.6?

See #191 (comment). A fix has been back-ported to python 3.7 but not 3.6 (https://bugs.python.org/issue37380). I'm not sure it's only AsyncMappingKernelManager that doesn't work though, or if it affects all kernel managers.

[kevin-bates]

Thanks for opening this issue Eric.

Since this appears to be a popen thing, I would expect it to affect any kernel manager that launches using popen - AsyncKernelManager, KernelManager, or BYO KM. Perhaps the async code path introduces a higher propensity to fail - I don't know.

David, do you have a traceback of an instance of this issue?

One way to spin this would be that we're asking Windows server administrators to run Jupyter Server on Python 3.7 if, and only if, they want to leverage the full capacity of Jupyter Server. That is, emphasize this as a server.

Existing Windows users, or those demanding that Python 3.6 (and even 3.5 for the sake of this argument), must settle for synchronous kernel management/client behaviors.

Could we flip the switch back to synchronous kernel management/client at runtime by interrogating the python version and platform and log a warning that their kernel management has been demoted until they run on 3.7+?

Since Kernel Providers are only async, I think we need to state a more strict policy, but that might be a while anyway and this issue is solved by time.

[echarles]

Could we flip the switch back to synchronous kernel management/client at runtime by interrogating the python version and platform and log a warning that their kernel management has been demoted until they run on 3.7+?

Sounds to me like a good tradeoff.

[blink1073]

Agreed

[davidbrochart]

I'm looking into it in #218.

[davidbrochart]

Strangely, I cannot make it fail anymore on Windows, even for python3.5.

[davidbrochart]

Given that it was always failing on python3.5 (and randomly on python3.6), I'm tempted to just allow these versions on Windows again, what do you think?

[Zsailer]

@davidbrochart Does this mean Jupyter Server works on 3.5 and above for all platforms? Is so, it's fine with me to support 3.5 for a while longer.

[davidbrochart]

That's what the tests say, but we have seen it has not always been the case. I don't know if we can close our eyes on it and take the risk to support 3.5 and above on all platforms. On the other hand maybe it was due to a particular situation that is not there anymore because the code has changed. I'd say that the fact that we don't see any problem on 3.5, which was always failing, tips the balance in favour of reenabling 3.5+ on Windows.

[blink1073]

I say we enable Py 3.6+ across the board, and add a retry step to the windows build if it gets flaky, considering it best effort.