Python: InvalidPythonEnvironment on interpreter startup

[isabelizimm]

Positron Version:

Positron dev

Steps to reproduce the issue:

1. start up any interpreter
2. check output

[Python] During handling of the above exception, another exception occurred:
[Python] 
[Python] Traceback (most recent call last):
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/pygls/protocol.py", line 400, in _handle_request
[Python]     self._execute_request(msg_id, handler, params)
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/pygls/protocol.py", line 322, in _execute_request
[Python]     self._send_response(msg_id, handler(params))
[Python]                                 ^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/positron_jedilsp.py", line 551, in positron_document_symbol
[Python]     return document_symbol(server, params)
[Python]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi_language_server/server.py", line 435, in document_symbol
[Python]     jedi_script = jedi_utils.script(server.project, document)
[Python]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi_language_server/jedi_utils.py", line 118, in script
[Python]     return Script(code=document.source, path=document.path, project=project)
[Python]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/__init__.py", line 119, in __init__
[Python]     self._inference_state = InferenceState(
[Python]                             ^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/inference/__init__.py", line 90, in __init__
[Python]     self.compiled_subprocess = environment.get_inference_state_subprocess(self)
[Python]                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/environment.py", line 106, in get_inference_state_subprocess
[Python]     return InferenceStateSubprocess(inference_state, self._get_subprocess())
[Python]                                                      ^^^^^^^^^^^^^^^^^^^^^^
[Python]   File "/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/environment.py", line 79, in _get_subprocess
[Python]     raise InvalidPythonEnvironment(
[Python] positron._vendor.jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/isabelzimmerman/.pyenv/versions/3.11.4/envs/docker/bin/python': InternalError("The subprocess /Users/isabelzimmerman/.pyenv/versions/3.11.4/envs/docker/bin/python has crashed (EOFError('Ran out of input'), stderr=).")
[Python] [positron._vendor.pygls.protocol] Sending data: {"error": {"code": -32603, "message": "positron._vendor.jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/isabelzimmerman/.pyenv/versions/3.11.4/envs/docker/bin/python': InternalError(\"The subprocess /Users/isabelzimmerman/.pyenv/versions/3.11.4/envs/docker/bin/python has crashed (EOFError('Ran out of input'), stderr=).\")", "data": "{'traceback': ['  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/pygls/protocol.py\", line 400, in _handle_request\\n    self._execute_request(msg_id, handler, params)\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/pygls/protocol.py\", line 322, in _execute_request\\n    self._send_response(msg_id, handler(params))\\n                                ^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/positron_jedilsp.py\", line 551, in positron_document_symbol\\n    return document_symbol(server, params)\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi_language_server/server.py\", line 435, in document_symbol\\n    jedi_script = jedi_utils.script(server.project, document)\\n                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi_language_server/jedi_utils.py\", line 118, in script\\n    return Script(code=document.source, path=document.path, project=project)\\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/__init__.py\", line 119, in __init__\\n    self._inference_state = InferenceState(\\n                            ^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/inference/__init__.py\", line 90, in __init__\\n    self.compiled_subprocess = environment.get_inference_state_subprocess(self)\\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/environment.py\", line 106, in get_inference_state_subprocess\\n    return InferenceStateSubprocess(inference_state, self._get_subprocess())\\n                                                     ^^^^^^^^^^^^^^^^^^^^^^\\n', '  File \"/Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/positron/_vendor/jedi/api/environment.py\", line 79, in _get_subprocess\\n    raise InvalidPythonEnvironment(\\n']}"}, "jsonrpc": "2.0", "id": 1}

I don't notice any effects from this error from a quick check, but it seems to occur on startup for me, but this seems related to recent vendoring.

[isabelizimm]

Seeing a different error printed out to the Console on startup:

Command failed: . /Users/isabelzimmerman/.pyenv/versions/3.11.4/envs/test311/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /Users/isabelzimmerman/code/positron/extensions/positron-python/pythonFiles/printEnvVariables.py
/Users/isabelzimmerman/.bashrc: line 5: nvm: command not found
Debugger listening on ws://127.0.0.1:61484/d80ae2f2-4f33-4eae-ad62-18baf24152cb
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.

The interpreter failed, but once I started up a new one, all was fine.

EDIT: this one was due to an error in my file, disregard. But it is worth it to note that terminal errors populate like this? I'm not sure if they should be suppressed, but I definitely thought it was related to the python-extension files since this is the only place I've seen it populate.

[jgutman]

Also in Output > Python Language Server, I'm seeing the following:

Message: positron._vendor.jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python': InternalError("The subprocess /Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python has crashed (EOFError('Ran out of input'), stderr=).")
  Code: -32603 
[object Object]

and in Output > Window

2024-01-25 15:04:55.749 [error] positron._vendor.jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python': InternalError("The subprocess /Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python has crashed (EOFError('Ran out of input'), stderr=)."): Error: positron._vendor.jedi.api.environment.InvalidPythonEnvironment: Could not get version information for '/Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python': InternalError("The subprocess /Users/jacquelinegutman/.pyenv/versions/3.10.11/bin/python has crashed (EOFError('Ran out of input'), stderr=).")
	at handleResponse (/Users/jacquelinegutman/positron/extensions/positron-python/node_modules/vscode-jsonrpc/lib/common/connection.js:565:48)
	at handleMessage (/Users/jacquelinegutman/positron/extensions/positron-python/node_modules/vscode-jsonrpc/lib/common/connection.js:345:13)
	at processMessageQueue (/Users/jacquelinegutman/positron/extensions/positron-python/node_modules/vscode-jsonrpc/lib/common/connection.js:362:17)
	at Immediate.<anonymous> (/Users/jacquelinegutman/positron/extensions/positron-python/node_modules/vscode-jsonrpc/lib/common/connection.js:334:13)
	at process.processImmediate (node:internal/timers:476:21)
	at process.callbackTrampoline (node:internal/async_hooks:130:17)

[isabelizimm]

This error occurs in any LSP subprocess that requires pickling, eg going to definition or viewing document symbols.

[Python] [positron._vendor.pygls.protocol.json_rpc] ERROR | Failed to handle request 86 textDocument/definition 

[seeM]

I briefly looked into this before I went OOO. Here are some findings which may be helpful.

This issue is related to the upstream issue: microsoft/vscode-python#22659.

I originally noticed that microsoft/vscode-python#22466 introduced the issue, so I reverted it in https://github.com/posit-dev/positron-python/pull/300.

But it looks like a better fix was made upstream in microsoft/vscode-python#22713. So maybe as a starting point we should revert my revert PR 300 and then cherry-pick the upstream PR 22713?

IIUC there is also an issue in Jedi's subprocess script which is expected to write a value to stdout that can be unpickled, but it's currently crashing.

I think this is due to the way our vendoring rewrites imports, clashing with how the above script works. It's not pretty at all, but I think we can fix it by updating our jedi.patch file to patch this function:

https://github.com/davidhalter/jedi/blob/740b474eda983f5b44573d1748587709e3116815/jedi/inference/compiled/subprocess/__main__.py#L11-L17

to:

def _get_paths():
    # Get the path to positron, in which jedi and parso are vendored.
    _d = os.path.dirname
    return {"positron": _d(_d(_d(_d(_d(_d(_d(__file__)))))))}

I don't yet fully understand why this script needs to hack with the import machinery, but I think patching it as above will make it function as it should given the rewritten vendored imports.

For reference, the way I added to patch files was to first stage the current version of the file git add <file> (may need a -f if it's git ignored), then make the change locally (sometimes in VIM or some other basic editor to avoid unnecessary changes e.g. auto-formatters), then git diff <file> >> <path-to-patch-file> to append the change to the patch file. I have also manually edited patch files occasionally. There is certainly a better way to do it so please share any tips!

Hope this helps.

[isabelizimm]

Ah, nice! This is super helpful context.

I narrowed down the impact of this error to the Positron LSP features that use jedi's Script under the hood, which is features like finding references, hovering, etc. You can hack past the InvalidPythonEnvironmentError by passing jedi's InterpreterEnvironment() as the environment in the Script, but the way it is set up right now would involve copying in each of those jedi features to our LSP to change one parameter, which feels a little gross and still has different errors. Also, that's just treating the symptoms, since I am not sure why the environment resolving differently--I don't think that it should be any different.

I can narrow down the issue to working at b7d17a and broken at 9ec300, so I'll start with cherry picking the upstream PR!

[jgutman]

After merge of https://github.com/posit-dev/positron-python/pull/342 I can no longer reproduce the original InvalidPythonEnvironment error.

We ran into a new error, where certain LSP actions in certain Python interpreters would result in the following error message TypeError: sequence item 0: expected str instance, Name found. However, this was only reproducible in certain environments. I repro'd in 1 interpreter, but not any of my others, and @seeM couldn't repro at all.

We tracked this down to incorrectly typing import names as parso.python.tree.Name rather than positron._vendor.parso.python.tree.Name in some cases, and given that this only occurs in certain interpreters, we think this may have to do with reading in cached pickled files from before vendoring was in place, giving the incorrect unvendored path. We confirmed that clearing out the old caches pkl files does make this error disappear.

Next step is to check if patching this: https://github.com/davidhalter/parso/blob/master/parso/cache.py#L52-L57
can help us avoid this cache issue

[juliasilge]

In dev Positron as of today, I can start up a Python interpreter and I see:

• no InvalidPythonEnvironment errors in the "Python" output
• no InvalidPythonEnvironment errors in the "Python Language Server" output
• no InvalidPythonEnvironment errors in the "Window" output
• no "TypeError: sequence item 0: expected str instance, Name found" errors in any of half a dozen Python interpreters I tried