Exit non-zero if no tests were run

[esiegerman]

[Split out of https://github.com//issues/500#issuecomment-112204804 into a separate issue]

If no tests were run, pytest should exit with a non-zero status.

[mostafahussein]

I think SKIPPED tests also should have a non-zero exit !

[nicoddemus]

@sh3llsh0ck you mean if a test session skipped all collected tests?

[mostafahussein]

if there is a skipped test , like when there is a failed test with other passed tests, the whole test exit with 1

in my case i have a custom option to pass parameter to test function, what if i forgot to pass this parameter ?

i saw the mentioned comment says the non-zero exit required for jenkins.
and in my case too when i am going to use jenkins, if i forgot to add a parameter test will be SKIPPED but jenkins will considered the test is passed (due to zero exit) which is not happened.

so the non-zero exit will make me revise jenkins configuration in this case to see if i forgot to pass a parameter or something

[nicoddemus]

"Skip" is meant for tests that are OK to be skipped under the current configuration. It is common for example to skip tests are specific to a platform when running in a different platform, or tests meant for a specific Python version to skip in other Python versions. In all those cases returning non-zero would break the test suite, which is not desirable.

For your specific case, wouldn't make more sense to make your test function to fail instead of skipping if the test does not receive the parameter?

[mostafahussein]

so if it is not OK , I have to add something to make it fail instead of skip, so i can make the test fail if the required python version or any required package version is not as expected, right ?

i have followed the documentation in order to add a custom option, but there is nothing about the status:

for example, why metafunction.parametrize does not have an option called if_missing default to skip?

Metafunc.parametrize(argnames, argvalues, indirect=False, ids=None, scope=None, if_missing='Skip')

or there is another way to make it fail when i forgot to pass a parameter that is required ?

[nicoddemus]

Can you post some minimum example of what you are trying to accomplish? Best create a new issue for that, as this is getting off-topic.

[ghostsquad]

I definitely agree that if no tests are run, that's a problem.

[nicoddemus]

I definitely agree that if no tests are run, that's a problem.

We will merge #817 soon with that. 😄

[ghostsquad]

@nicoddemus - Your profile picture is the best. I love those penguins. Thanks for the quick response!