Pytest 7.4.0 release broke addoption behaviour

[enrico-usai]

Hi!

In the AWS ParallelCluster project we are using pytest and adding some custom options to it:

• https://github.com/aws/aws-parallelcluster/blob/develop/tests/integration-tests/conftest.py#L109
• https://github.com/aws/aws-parallelcluster/blob/develop/tests/integration-tests/test_runner.py#L466

Our tests started to fail after the automatic upgrade to 7.4.0 with an error like:

2023-06-23 12:41:56,473 - INFO - test_runner - Starting pytest in region ... with params ['-s', '-vv', '-l', '--tests-log-file=..., '--output-dir=...', '--key-name=...', '--key-path=...', ...]
...
ERROR: usage: test_runner.py [options] [file_or_dir] [file_or_dir] [...]
test_runner.py: error: unrecognized arguments: --tests-log-file=... --output-dir=... --key-name=... --key-path=... --stackname-suffix ...

So we had to pin the version to 7.3.2: aws/aws-parallelcluster#5425

Pip freeze

Installed packages:
Package                             Version
----------------------------------- -----------
assertpy                            1.1
attrs                               23.1.0
aws-cdk.assets                      1.204.0
aws-cdk.aws-acmpca                  1.204.0
aws-cdk.aws-apigateway              1.204.0
aws-cdk.aws-applicationautoscaling  1.204.0
aws-cdk.aws-autoscaling             1.204.0
aws-cdk.aws-autoscaling-common      1.204.0
aws-cdk.aws-autoscaling-hooktargets 1.204.0
aws-cdk.aws-batch                   1.204.0
aws-cdk.aws-certificatemanager      1.204.0
aws-cdk.aws-cloudformation          1.204.0
aws-cdk.aws-cloudfront              1.204.0
aws-cdk.aws-cloudwatch              1.204.0
aws-cdk.aws-codebuild               1.204.0
aws-cdk.aws-codecommit              1.204.0
aws-cdk.aws-codeguruprofiler        1.204.0
aws-cdk.aws-codestarnotifications   1.204.0
aws-cdk.aws-cognito                 1.204.0
aws-cdk.aws-dynamodb                1.204.0
aws-cdk.aws-ec2                     1.204.0
aws-cdk.aws-ecr                     1.204.0
aws-cdk.aws-ecr-assets              1.204.0
aws-cdk.aws-ecs                     1.204.0
aws-cdk.aws-efs                     1.204.0
aws-cdk.aws-elasticloadbalancing    1.204.0
aws-cdk.aws-elasticloadbalancingv2  1.204.0
aws-cdk.aws-events                  1.204.0
aws-cdk.aws-fsx                     1.204.0
aws-cdk.aws-globalaccelerator       1.204.0
aws-cdk.aws-iam                     1.204.0
aws-cdk.aws-imagebuilder            1.204.0
aws-cdk.aws-kinesis                 1.204.0
aws-cdk.aws-kms                     1.204.0
aws-cdk.aws-lambda                  1.204.0
aws-cdk.aws-logs                    1.204.0
aws-cdk.aws-route53                 1.204.0
aws-cdk.aws-route53-targets         1.204.0
aws-cdk.aws-s3                      1.204.0
aws-cdk.aws-s3-assets               1.204.0
aws-cdk.aws-sam                     1.204.0
aws-cdk.aws-secretsmanager          1.204.0
aws-cdk.aws-servicediscovery        1.204.0
aws-cdk.aws-signer                  1.204.0
aws-cdk.aws-sns                     1.204.0
aws-cdk.aws-sns-subscriptions       1.204.0
aws-cdk.aws-sqs                     1.204.0
aws-cdk.aws-ssm                     1.204.0
aws-cdk.aws-stepfunctions           1.204.0
aws-cdk.cloud-assembly-schema       1.204.0
aws-cdk.core                        1.204.0
aws-cdk.custom-resources            1.204.0
aws-cdk.cx-api                      1.204.0
aws-cdk.region-info                 1.204.0
aws-parallelcluster                 3.7.0
bcrypt                              4.0.1
boto3                               1.26.159
botocore                            1.29.159
cattrs                              23.1.2
certifi                             2023.5.7
cffi                                1.15.1
cfn-flip                            1.3.0
charset-normalizer                  3.1.0
click                               8.1.3
clickclick                          20.10.2
connexion                           2.13.1
constructs                          3.4.332
contourpy                           1.1.0
cryptography                        41.0.1
cycler                              0.11.0
defusedxml                          0.7.1
docopt                              0.6.2
exceptiongroup                      1.1.1
execnet                             1.9.0
fabric                              2.6.0
filelock                            3.12.2
Flask                               2.2.5
fonttools                           4.40.0
future                              0.18.3
idna                                3.4
importlib-metadata                  6.7.0
importlib-resources                 5.12.0
inflection                          0.5.1
iniconfig                           2.0.0
invoke                              1.7.3
itsdangerous                        2.1.2
Jinja2                              3.1.2
jmespath                            0.10.0
jsii                                1.84.0
jsonschema                          4.17.3
junitparser                         3.1.0
kiwisolver                          1.4.4
MarkupSafe                          2.1.3
marshmallow                         3.19.0
matplotlib                          3.7.1
numpy                               1.25.0
packaging                           23.1
paramiko                            3.2.0
pathlib2                            2.3.7.post1
pcluster-client                     1.0.0
pexpect                             4.8.0
Pillow                              9.5.0
pip                                 23.1.2
pluggy                              1.2.0
psutil                              5.9.5
ptyprocess                          0.7.0
publication                         0.0.3
py                                  1.11.0
pycparser                           2.21
pykwalify                           1.8.0
PyNaCl                              1.5.0
pyOpenSSL                           23.2.0
pyparsing                           3.1.0
pyrsistent                          0.19.3
pytest                              7.4.0
pytest-datadir                      1.4.1
pytest-html                         3.2.0
pytest-metadata                     3.0.0
pytest-rerunfailures                11.1.2
pytest-sugar                        0.9.7
pytest-xdist                        3.3.1
python-dateutil                     2.8.2
PyYAML                              5.4.1
requests                            2.31.0
retrying                            1.3.4
ruamel.yaml                         0.17.32
ruamel.yaml.clib                    0.2.7
s3transfer                          0.6.1
setuptools                          58.1.0
six                                 1.16.0
tabulate                            0.8.10
termcolor                           2.3.0
tomli                               2.0.1
troposphere                         4.3.2
typeguard                           2.13.3
typing_extensions                   4.6.3
untangle                            1.2.1
urllib3                             1.26.16
Werkzeug                            2.3.6
xmltodict                           0.13.0
zipp                                3.15.0

[bluetech]

Hi @enrico-usai,

How do you invoke pytest?

[bluetech]

Sorry, I see that you linked to it. I'll try and have a look.

[bluetech]

If I am reading correctly (from a quick skim in github), it looks like you are running with --rootdir=./tests. I suspect this is causing the issue; basically the conftest.py file is outside of the rootdir so it's not found any more (see #11043, also mentioned in the changelog). The more "correct" rootdir is the aws-parallelcluster/tests/integration-tests/ path where all of the config files etc. are located. Can you try changing to that?

[enrico-usai]

Thank you @bluetech for your analysis!

I'll try to fix this.

[greschd]

FWIW, in our case this was solved by adding an empty [tool.pytest.ini_options] to pyproject.toml so that pytest recognizes the rootdir.

As far as I can tell, in your case adding [pytest] to https://github.com/aws/aws-parallelcluster/blob/develop/tests/integration-tests/tox.ini might work.

This is the relevant part of the docs: https://docs.pytest.org/en/stable/reference/customize.html#finding-the-rootdir
Specifically:

Files will only be matched for configuration if:

• pytest.ini: will always match and take precedence, even if empty.
• pyproject.toml: contains a [tool.pytest.ini_options] table.
• tox.ini: contains a [pytest] section.
• setup.cfg: contains a [tool:pytest] section.

[github-actions]

This issue is stale because it has been open for 14 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.