Coverage report generation incompatible with pytest `tmpdir` feature

[bluenote10]

Given a project structure like:

myproject
├── myproject
│   └── __init__.py
├── tests
│   └── myproject
│       └── test.py
└── setup.cfg

I'm running the command in the toplevel myproject directory:

py.test --cov=myproject --cov-report html

On stdout I can see the log output:

Coverage HTML written to dir htmlcov

However no output has been written, i.e., the myproject directory does not contain htmlcov (and neither of its subdirectories). Even a find / -name "*htmlcov*" reveals nothing.

Notes:

• The output is written if I use --cov-report html:/absolute/path/to/myproject/htmlcov.
• I get the same issue for --cov-report xml.
• My tests chdir a lot -- can this mess up pytest-cov?

Versions:

• pytest: 3.8.1
• pytest-cov: 2.6.0

[blueyed]

My tests chdir a lot -- can this mess up pytest-cov?

Sounds like it. I assume it gets written to some temporary directory maybe (outside of with you run find then).
Try exporting COV_DEBUG (see coverage.py doc for options) to see what is going on.

[blueyed]

Is .coverage created as expected?
I.e. does coverage report work after running the tests?
Then coverage html htmlcov might work also from there.

[bluenote10]

Yes the .coverage file gets created in the current directory and I also get sensible output from the terminal reporters. When running coverage html htmlcov afterwards in the project directory I get the error:

No source for code: '/home/me/workspace/myproject/htmlcov': [Errno 21] Is a directory: u'/home/me/workspace/myproject/htmlcov'

This at least creates the htmlcov folder in the project directory, but it is empty.

I haven't gotten much out of exporting COV_DEBUG so far.

[blueyed]

The command should be coverage html -d htmlcov, of course - sorry.

I haven't gotten much out of exporting COV_DEBUG so far.

What options are you using?

[blueyed]

btw: it is probably better to look at pytest-cov's code, where it handles the html report - i.e. put a pdb.set_trace() there / print os.getcwd().

[bluenote10]

The command should be coverage html -d htmlcov, of course - sorry.

Ah yes, this generates the output just fine.

What options are you using?

What I did was just export COV_DEBUG=true because I couldn't find any documentation on COV_DEBUG. I also don't have a coverage.py.

[blueyed]

coverage.py is https://github.com/nedbat/coveragepy/

The environment variable is COVERAGE_DEBUG (sorry), and you could try COVERAGE_DEBUG=process,config.

[bluenote10]

Debugging in this line showed that

• self.cov_report['html'] is None so it will not try to write to an explicitly given directory
• os.getcwd() returns something like /tmp/pytest-of-username/pytest-66/test_blah0 which is a standard directory using pytests tmpdir feature. This directory will get deleted immediately after running the tests, i.e., the report gets deleted as well.

So the question is: What is the nicest way to recover the working directory? Just store it initially and os.chdir back when writing the output? Or should pytest reset the working directory instead?

[blueyed]

Are you changing the directory yourself?
Normally you would use with tmpdir.as_cwd: …, which restores it automatically.

[blueyed]

I assume pytest-cov could make the path absolute to the session's initial cwd maybe, and/or write the report with changing the dir itself temporarily (to the initial dir).

[blueyed]

pytest itself has problems with this during collection already btw (pytest-dev/pytest#4317).

[bluenote10]

Are you changing the directory yourself? Normally you would use with tmpdir.as_cwd: …, which restores it automatically.

Yes it turned out there was a test which sets the working directory manually.

[ionelmc]

We could try to make sure we dump the report in the original cwd tho (just like we handle config paths).

[blueyed]

There is invocation_dir on the pytest config object that could be used.