Sys.Path missing some directories in latest 3.0.17 version

[vinsworldcom]

Unless I'm missing something, the full and full-plugin-admin 64-bit versions both do not include the:

• plugins\Config\PythonScript\lib'
• plugins\Config\PythonScript\scripts'

directories in sys.path. I was previously using 3.0.16 and that one did:

print(sys.path)

['C:\\usr\\bin\\npp64\\plugins\\PythonScript\\lib'
 'C:\\usr\\bin\\npp64\\plugins\\Config\\PythonScript\\lib'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\scripts'
 'C:\\usr\\bin\\npp64\\plugins\\Config\\PythonScript\\scripts'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\lib\\lib-tk'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\python310.zip'
 'C:\\Users\\VinsWorldcom\\AppData\\Local\\Programs\\Python\\Python310\\Lib'
 'C:\\Users\\VinsWorldcom\\AppData\\Local\\Programs\\Python\\Python310\\DLLs'
 'C:\\usr\\bin\\npp64'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python310\\site-packages'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python310\\site-packages\\win32'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python310\\site-packages\\win32\\lib'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python310\\site-packages\\Pythonwin'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\lib\\site-packages']

by contrast, the new 3.0.17 sys.path:

['C:\\usr\\bin\\npp64\\plugins\\PythonScript\\python312.zip'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\Lib'
 'C:\\usr\\bin\\npp64'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python312\\site-packages'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python312\\site-packages\\win32'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python312\\site-packages\\win32\\lib'
 'C:\\Users\\VinsWorldcom\\AppData\\Roaming\\Python\\Python312\\site-packages\\Pythonwin'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\Lib\\site-packages']

Cheers.

[chcg]

Seems something changed in the init phase of python. Maybe https://docs.python.org/3/c-api/init_config.html#c.PyConfig_Read

Changed in version 3.11: PyConfig_Read() no longer calculates all paths, and so fields listed under Python Path Configuration may no longer be updated until Py_InitializeFromConfig() is called.

Needs further investigation how to get back the previous behaviour.

[vinsworldcom]

@chcg this commit makes it even worse:

f4b6f02

>>> print(sys.path)
['C:\\usr\\bin\\npp64\\plugins\\PythonScript\\python312.zip', 'C:\\usr\\bin\\npp64\\plugins\\PythonScript', 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\Lib', 'C:\\usr\\bin\\npp64']

Cheers.

[chcg]

See https://docs.python.org/3/c-api/init_config.html#init-path-config and https://docs.python.org/3/c-api/init_config.html#c.Py_InitializeFromConfig.

If at least one “output field” is not set, Python calculates the path configuration to fill unset fields. If module_search_paths_set is equal to 0, module_search_paths is overridden and module_search_paths_set is set to 1.

It is possible to completely ignore the function calculating the default path configuration by setting explicitly all path configuration output fields listed above. A string is considered as set even if it is non-empty. module_search_paths is considered as set if module_search_paths_set is set to 1. In this case, module_search_paths will be used without modification.

PS code prepends or appends path to config.module_search_paths which gets overridden by call Py_InitializeFromConfig as
config.module_search_paths_set = 1;
is not set. The drawback if this is done just the configured paths from PS are used for sys.path and no calculated ones.

So probably it is necessary to set our additional paths after Py_InitializeFromConfig via

  /* If you want to modify the default set of paths, finish
       initialization first and then use PySys_GetObject("path") */

[chcg]

@vinsworldcom Could you test that this build from
https://ci.appveyor.com/project/bruderstein/pythonscript/builds/48895630/job/r03qqepvra2o1r3m/artifacts
is ok also in your setup.

[vinsworldcom]

@chcg it's better, but now not finding the PythonScript site directory:

'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\lib'
 'C:\\usr\\bin\\npp64\\plugins\\Config\\PythonScript\\lib'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\scripts'
 'C:\\usr\\bin\\npp64\\plugins\\Config\\PythonScript\\scripts'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\lib\\lib-tk'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\python312.zip'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript'
 'C:\\usr\\bin\\npp64\\plugins\\PythonScript\\Lib'
 'C:\\usr\\bin\\npp64']

Notice :

'C:\usr\bin\npp64\plugins\PythonScript\lib\site-packages

is missing and it was there before (see previous posts).

Also, now it's not finding any of the local system Python paths. Note I do not have the "Prefer installed Python libraries ..." checked, but the previous versions always found the local system Python paths:

'C:\Users\VinsWorldcom\AppData\Roaming\Python\Python3xx\site-packages\**'

This new commit is no longer finding them, and as you can see from above, the "original" 3.0.17 did find the local system Python paths.

Cheers.

[vinsworldcom]

At the very least, if it imports the local PythonScript site packages:

plugins\PythonScript\lib\site-packages

Then I can put a .pth file in that directory with the paths of my system Python installation site files and get the functionality I want.

Cheers.