Package scoped fixture will not execute teardown

[hennadii-demchenko]

I am experiencing a weird issue:
'package' scoped fixture will not execute teardown after last test in package is completed if afformentioned fixture is defined outside of package

Environment

~ python --version
Python 3.8.5

~ pip freeze
allure-pytest==2.8.24
allure-python-commons==2.8.24
assertpy==1.1
attrdict==2.0.1
attrs==20.3.0
bcrypt==3.2.0
cffi==1.14.4
cryptography==3.2.1
cycler==0.10.0
future==0.18.2
iniconfig==1.1.1
kiwisolver==1.3.1
matplotlib==3.3.3
mininet==2.3.0.dev6
more-itertools==8.6.0
numpy==1.19.4
packaging==20.7
paramiko==2.7.2
pexpect==4.8.0
pexpect-serial==0.1.0
Pillow==8.0.1
pluggy==0.13.1
ptyprocess==0.6.0
py==1.9.0
pycparser==2.20
PyNaCl==1.4.0
pyparsing==2.4.7
Pypubsub==4.0.3
pyserial==3.5
pytest==6.2.1
pytest-dependency==0.5.1
pytest-logger==0.5.1
pytest-sugar==0.9.4
python-dateutil==2.8.1
PyYAML==5.3.1
singleton-decorator==1.0.0
six==1.15.0
termcolor==1.1.0
toml==0.10.2
wcwidth==0.2.5

~ uname -a 
5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Example to reproduce

~ tree test
test
├── __init__.py
├── conftest.py
├── sub1
│   └── __init__.py
│   └── test_foo.py
└── sub2
    └── __init__.py
    └── test_bar.py
2 directories, 3 files

~ cat test/conftest.py
import pytest

@pytest.fixture(scope='package')
def package_scoped():
    print('setup')
    yield
    print('teardown')

~ cat test/sub1/test_foo.py
def test_foo(package_scoped): pass

~ cat test/sub2/test_bar.py
def test_foo(package_scoped): pass

Result

~ pytest test --setup-plan -p no:sugar
=============== test session starts ===============
platform linux -- Python 3.8.5, pytest-6.2.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/hdemchenko/git/uns
plugins: dependency-0.5.1, logger-0.5.1, allure-pytest-2.8.24
collected 2 items

test/sub1/test_foo.py
  SETUP    P package_scoped
        test/sub1/test_foo.py::test_foo (fixtures used: package_scoped)
test/sub2/test_bar.py
        test/sub2/test_bar.py::test_foo (fixtures used: package_scoped)
  TEARDOWN P package_scoped

=============== no tests ran in 0.01s ===============

If I create two conftest files in each package with the package scoped fixture I intend to use then I can see it behaving as I was expecting.
Example:

~ mv test/conftest.py test/sub1/
~ cp test/sub1/conftest.py test/sub2/
~ pytest test --setup-plan -p no:sugar
=============== test session starts ===============
platform linux -- Python 3.8.5, pytest-6.2.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/hdemchenko/git/uns
plugins: dependency-0.5.1, logger-0.5.1, allure-pytest-2.8.24
collected 2 items

test/sub1/test_foo.py
  SETUP    P package_scoped
        test/sub1/test_foo.py::test_foo (fixtures used: package_scoped)
  TEARDOWN P package_scoped
test/sub2/test_bar.py
  SETUP    P package_scoped
        test/sub2/test_bar.py::test_foo (fixtures used: package_scoped)
  TEARDOWN P package_scoped

=============== no tests ran in 0.01s ===============

[bluetech]

Just to make sure I understand, what you expect is that the first example have the same behavior as the second?

You have 3 packages in play: test, test.sub1, test.sub2. In the first example the fixture's scope is package test. In the second example the fixtures' scopes are test.sub1 and test.sub2.

[hennadii-demchenko]

@Zac-HD, so should I understand package fixture scope such as:

• starts before the first test requiring the fixture
• ends with the last test within package scope
• package scope is literally the package fixture was defined in

To be honest I am a little confuesd with your explanation because every other scope, except of maybe session has different behavior.
For example what should happen if:

• there is a single module scoped fixture in conftest file at test-root directory named say 'my_fixture'
• there is some amount of packages in same test-root directory
• there is some amount of test modules in each package
• there are some tests in each module and each of them require the same 'my_fixture' fixture

What would setup/teardown plan look like in this case? And to answer your question about my expectation - i kinda expect the same setup/teardown behavior to be applied to package level scopes, with exception that the boundary is on the package but not the module.

[hennadii-demchenko]

And to proove my point further - another way to get expected (at least by me) behavior is:

• have test directory structure

~ tree test
test
├── __init__.py
├── conftest.py
├── sub1
│   └── __init__.py
│   └── conftest.py
│   └── test_foo.py
└── sub2
    └── __init__.py
    └── conftest.py
    └── test_bar.py

• leave test/conftest.py the same as in report
• have test/sub1/conftest.py and test/sub2/conftest.py simply import fixture and nothing else

[hennadii-demchenko]

@Zac-HD I think the best way to summarize is that currently package scope is related to "the package" when I expect it to be for "a package"

[bluetech]

Thanks for the details @hennadii-demchenko. I agree with your intuition here. I think the package scope and/or conftest handling is quite buggy. There's also some related discussion in #7777 (in the sense that it affects the behavior here).

I'm marking this as a bug. Personally I do plan to dig into this more soon.

[hennadii-demchenko]

@bluetech, thanks I'll keep an eye on the progress.

Here's a quick tip on workaround for other people who might stumble upon this issue (it is easy although not very beatiful)
All you need to do is:

• create conftest file in each package you want to use the fixture
• just import the fixture function in conftest file

[hennadii-demchenko]

@bluetech any luck on getting back to this issue?

[RonnyPfannschmidt]

to add extra context, each import of a fixture creates a new definition point, thus iporting the fixtures to closer file location points fixes the issues by creating new definitions

its not clear to me whats the exact scoping of package scope fixtures, as they can have multiple points of reference its something tricky to get right

[hennadii-demchenko]

Hey, it's been a while. Any chance this is gonna be fixed at all?

[stormi]

I just met this exact issue.