Issues with nodeid consistency when pytest --pyargs is not run from the root directory #3714
Description
Consider the following (standard) test layout:
quux
└── pkg
├── __init__.py
└── tests
├── conftest.py
├── __init__.py
└── test_foo.py
where quux is in PYTHONPATH (e.g., add it directly to PYTHONPATH, or quux could actually be site-packages).
conftest.py is set to report the items nodeids:
def pytest_collection_modifyitems(session, config, items):
for item in items:
print(item.nodeid)
and test_foo contains a single test, test_bar
If one runs pytest --pyargs pkg, --pyargs pkg.tests, or pkg.tests.test_foo, from quux, then the nodeid is correctly reported as pkg/tests/test_foo.py::test_bar
If one runs the same commands from quux's parent, then the nodeids are reported as quux/tests/test_foo.py::test_bar -- i.e., relative to cwd rather than to where the tests have been found in the PYTHONPATH.
Finally, if one runs the same commands from a directory that is unrelated (not a parent) of quux, then the nodeids are reported as pkg/tests/test_foo.py::test_bar, tests/test_foo.py::test_bar, and test_foo.py::test_bar respectively -- i.e., relative to the parent of the last specified module.
I think the correct behavior is clearly to always report as relative to where the tests have been found in the PYTHONPATH, i.e. pkg/tests/test_foo.py::test_bar.
Tested from a fresh Py3.6 (Arch Linux repo Python) venv with just pytest 3.6.3 installed.
Seems related to #2775.