Not using `indirect` in `parametrize` causes module-scoped fixtures to be called multiple times

[guyboltonking]

In the following example:

import pytest

def p(s, *args):
    s = s.format(*args)
    print()
    print(">>> " + s)
    return s


calls = set()
def assert_called_only_once(result):
    global calls
    assert result not in calls
    calls.add(result)
    return result


# I'd expect this only to be called once, but it isn't:

@pytest.fixture(scope="module")
def fixture_with_params(fixture_param):
    return assert_called_only_once(p('fixture_with_params({})', fixture_param))

@pytest.mark.parametrize("param", [1, 2], scope="module")
@pytest.mark.parametrize("fixture_param", [3], scope="module")
def test_fixture_with_params(fixture_with_params, param):
    p('test_fixture_with_params({}, {})', fixture_with_params, param)


# Using indirect does work however:

@pytest.fixture(scope="module")
def fixture_with_indirect_params(request):
    return assert_called_only_once(p('fixture_with_indirect_params({})', request.param))

@pytest.mark.parametrize("param", [1, 2], scope="module")
@pytest.mark.parametrize("fixture_with_indirect_params", [3], scope="module", indirect=True)
def test_fixture_with_indirect_params(fixture_with_indirect_params, param):
    p('test_fixture_with_indirect_params({}, {})', fixture_with_indirect_params, param)

...I'd expect both tests to pass. However:

pytest tests/cli/fixture_params_test.py
======================================================================= test session starts =======================================================================
platform darwin -- Python 2.7.11, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /Users/guy/Work/scratch, inifile:
collected 4 items

fixture_params_test.py ...E                                                                                                                       [100%]

============================================================================= ERRORS ==============================================================================
_________________________________________________________ ERROR at setup of test_fixture_with_params[3-2] _________________________________________________________

fixture_param = 3

    @pytest.fixture(scope="module")
    def fixture_with_params(fixture_param):
>       return assert_called_only_once(p('fixture_with_params({})', fixture_param))

tests/cli/fixture_params_test.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = 'fixture_with_params(3)'

    def assert_called_only_once(result):
        global calls
>       assert result not in calls
E       AssertionError: assert 'fixture_with_params(3)' not in set(['fixture_with_indirect_params(3)', 'fixture_with_params(3)'])

tests/cli/fixture_params_test.py:13: AssertionError
---------------------------------------------------------------------- Captured stdout setup ----------------------------------------------------------------------
()
>>> fixture_with_params(3)
================================================================ 3 passed, 1 error in 0.06 seconds ================================================================

If test_fixture_with_params is supposed to call the fixture twice, I think this may be at least a documentation issue, since the behaviour is surprising?

Alternatively, if this form of parameterizing fixtures is incorrect, should it be an error?

Or should fixture_with_params only have been called once, and this is a bug?

$ pip list
Package        Version
-------------- -------
atomicwrites   1.1.5
attrs          18.1.0
funcsigs       1.0.2
more-itertools 4.2.0
pip            10.0.1
pluggy         0.6.0
py             1.5.4
pytest         3.6.3
setuptools     40.0.0
six            1.11.0
wheel          0.31.1
$ python --version
Python 2.7.11
$ uname -a
Darwin clubmoss.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

[pytestbot]

GitMate.io thinks possibly related issues are #447 (Fixture params not accessible inside the fixture, not getting called multiple times.), #1878 (session scoped fixtures get executed multiple times. I suspect I am using pytest wrong), #2704 (capsys fixture cannot be used with a session-scoped fixture), #2902 (Teardown higher-scoped fixtures when not used), and #2495 (Multiple invocations of class-scoped fixture change called finalizer order ).

[nicoddemus]

Hi @guyboltonking,

Thanks for the detailed report, we appreciate it.

Alternatively, if this form of parameterizing fixtures is incorrect, should it be an error?

I think it should raise an error given that fixture_param doesn't even exist as a fixture in your first example. @RonnyPfannschmidt what do you think?

[bluetech]

This was fixed by @aklajnert in #5798 (pytest 5.1.2).