Permission denied on Windows: EGG-INFO (regression in pluggy 0.12)

[nicoddemus]

Howdy,

Unfortunately it seems we still have a few glitches regarding importlib-metadata.

I'm getting this error on Windows while trying to run pytest:

λ pytest test_esss_jenkins.py
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\bruno.ESSS.CO\err-jenkins\.env37\Scripts\pytest.exe\__main__.py", line 5, in <module>
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\pytest.py", line 7, in <module>
    from _pytest.assertion import register_assert_rewrite
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\_pytest\assertion\__init__.py", line 12, in <module>
    from _pytest.assertion import rewrite
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\_pytest\assertion\rewrite.py", line 23, in <module>
    from _pytest.assertion import util
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\_pytest\assertion\util.py", line 10, in <module>
    import _pytest._code
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\_pytest\_code\__init__.py", line 6, in <module>
    from .code import Code  # noqa
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\_pytest\_code\code.py", line 14, in <module>
    import pluggy
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\pluggy\__init__.py", line 16, in <module>
    from .manager import PluginManager, PluginValidationError
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\pluggy\manager.py", line 6, in <module>
    import importlib_metadata
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\importlib_metadata\__init__.py", line 463, in <module>
    __version__ = version(__name__)
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\importlib_metadata\__init__.py", line 430, in version
    return distribution(package).version
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\importlib_metadata\__init__.py", line 220, in version
    return self.metadata['Version']
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\importlib_metadata\__init__.py", line 213, in metadata
    or self.read_text('')
  File "c:\users\bruno.esss.co\err-jenkins\.env37\lib\site-packages\importlib_metadata\__init__.py", line 390, in read_text
    return self._path.joinpath(filename).read_text(encoding='utf-8')
  File "C:\Program Files\Python37\lib\pathlib.py", line 1199, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "C:\Program Files\Python37\lib\pathlib.py", line 1186, in open
    opener=self._opener)
  File "C:\Program Files\Python37\lib\pathlib.py", line 1039, in _opener
    return self._accessor.open(self, flags, mode)
PermissionError: [Errno 13] Permission denied: 'c:\\users\\bruno.esss.co\\err-jenkins\\.env37\\EGG-INFO'

Steps to reproduce:

λ git clone git@github.com:ESSS/err-jenkins.git
λ cd err-jenkins\
λ py -m venv .env37
λ .env37\Scripts\activate
(.env37) λ python -m pip install -U pip
(.env37) λ pip install -r dev-requirements.txt
(.env37) λ pytest test_esss_jenkins.py

Running pip install pluggy==0.11 fixes the issue.

Any hints @asottile? 🤔

cc @dawelter2

[nicoddemus]

pip list in case it is useful:

(.env37) λ pip list
Package                 Version
----------------------- -----------
ansi                    0.1.3
asn1crypto              0.24.0
atomicwrites            1.3.0
attrs                   19.1.0
beautifulsoup4          4.7.1
certifi                 2019.3.9
cffi                    1.12.3
chardet                 3.0.4
Click                   7.0
colorama                0.4.1
colorlog                4.0.2
cryptography            2.6.1
errbot                  6.0.0
Flask                   1.0.3
idna                    2.8
importlib-metadata      0.15
itsdangerous            1.1.0
Jinja2                  2.10.1
Markdown                2.6.11
MarkupSafe              1.1.1
more-itertools          7.0.0
pip                     19.1.1
pluggy                  0.12.0
py                      1.8.0
pycparser               2.19
Pygments                2.4.2
pygments-markdown-lexer 0.1.0.dev39
pyOpenSSL               19.0.0
pytest                  4.5.0
pytest-mock             1.10.4
requests                2.22.0
rocket-python           1.2.8
setuptools              40.8.0
six                     1.12.0
soupsieve               1.9.1
urllib3                 1.25.3
waitress                1.3.0
wcwidth                 0.1.7
WebOb                   1.8.5
WebTest                 2.0.33
Werkzeug                0.15.4
zipp                    0.5.1

[nicoddemus]

(I don't doubt this is Windows specific, let me know if you want me to run some commands to help debug this)

[asottile]

pygments-markdown-lexer is putting a EGG-INFO directory in the prefix -- I think a fix needs to happen there

$ ls -al .env37/
total 24
drwxr-xr-x 5 asottile asottile 4096 May 29 13:33 .
drwxr-xr-x 5 asottile asottile 4096 May 29 13:33 ..
drwxr-xr-x 2 asottile asottile 4096 May 29 13:33 bin
drwxr-xr-x 2 asottile asottile 4096 May 29 13:31 include
drwxr-xr-x 3 asottile asottile 4096 May 29 13:31 lib
lrwxrwxrwx 1 asottile asottile    3 May 29 13:31 lib64 -> lib
-rw-r--r-- 1 asottile asottile   69 May 29 13:31 pyvenv.cfg
$ pip install pygments-markdown-lexer
Collecting pygments-markdown-lexer
  Using cached https://files.pythonhosted.org/packages/c3/12/674cdee66635d638cedb2c5d9c85ce507b7b2f91bdba29e482f1b1160ff6/pygments-markdown-lexer-0.1.0.dev39.zip
Requirement already satisfied: Pygments>=2.0 in ./.env37/lib/python3.7/site-packages (from pygments-markdown-lexer) (2.4.2)
Installing collected packages: pygments-markdown-lexer
  Running setup.py install for pygments-markdown-lexer ... done
Successfully installed pygments-markdown-lexer-0.1.0.dev39
You are using pip version 19.0.3, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ ls -al .env37/
total 28
drwxr-xr-x 6 asottile asottile 4096 May 29 13:34 .
drwxr-xr-x 5 asottile asottile 4096 May 29 13:33 ..
drwxr-xr-x 2 asottile asottile 4096 May 29 13:33 bin
drwxr-xr-x 2 asottile asottile 4096 May 29 13:34 EGG-INFO
drwxr-xr-x 2 asottile asottile 4096 May 29 13:31 include
drwxr-xr-x 3 asottile asottile 4096 May 29 13:31 lib
lrwxrwxrwx 1 asottile asottile    3 May 29 13:31 lib64 -> lib
-rw-r--r-- 1 asottile asottile   69 May 29 13:31 pyvenv.cfg

[asottile]

CC @jhermann (via https://github.com/jhermann/pygments-markdown-lexer)

[asottile]

there also appears to be something weird about PYTHONPATH to trigger this, I can't trigger it on posix without having PYTHONPATH=.env37 (which is wrong I think?)

[nicoddemus]

Not sure about that, I've used the builtin activation script, and don't have anything else in the PYTHONPATH... 🤔

[asottile]

spinning up my windows machine right now to test there :)

Here's the error I get in posix though:

$ PYTHONPATH=.env37 python
Python 3.7.3 (default, Mar 26 2019, 00:55:50) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/t/err-jenkins/.env37/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 463, in <module>
    __version__ = version(__name__)
  File "/tmp/t/err-jenkins/.env37/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 430, in version
    return distribution(package).version
  File "/tmp/t/err-jenkins/.env37/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 220, in version
    return self.metadata['Version']
  File "/tmp/t/err-jenkins/.env37/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 213, in metadata
    or self.read_text('')
  File "/tmp/t/err-jenkins/.env37/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 390, in read_text
    return self._path.joinpath(filename).read_text(encoding='utf-8')
  File "/usr/lib/python3.7/pathlib.py", line 1199, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/usr/lib/python3.7/pathlib.py", line 1186, in open
    opener=self._opener)
IsADirectoryError: [Errno 21] Is a directory: '/tmp/t/err-jenkins/.env37/EGG-INFO'
>>> 

[asottile]

really weird! -mvenv virtualenvs put the root of the virtaulenv on sys.path:

>>> import sys
>>> sys.path
['', 'C:\\Python37\\python37.zip', 'C:\\Python37\\DLLs', 'C:\\Python37\\lib', 'C
:\\Python37', 'C:\\Users\\IEUser\\AppData\\Local\\Temp\\t\\venv', 'C:\\Users\\IE
User\\AppData\\Local\\Temp\\t\\venv\\lib\\site-packages']

[asottile]

created https://bugs.python.org/issue37089 since that seems pretty broken!

[asottile]

seems importlib-metadata is already catching some of these errors, just needs to catch one more -- crafting a fix for that right now!

[asottile]

https://gitlab.com/python-devs/importlib_metadata/merge_requests/72 for the fix there

[nicoddemus]

Nice, and it seems it has been published. Awesome, thanks!

We can close this now. 👍

[asottile]

Had to follow up with one more for windows, so once that gets merged it'll be on all platforms