missing files on test

[marianielias]

Hi, testing with "pytest pandas" from a pypi package returns a bunch of errors related to missing files.
I understand that those files are removed on propose ?
Shouldn't the tests skip the use of those files if they aren't present ?

[TomAugspurger]

What version of pandas?

[marianielias]

v0.23.4

The MANIFEST.in removes the csv files for example.
https://github.com/pandas-dev/pandas/blob/v0.23.4/MANIFEST.in#L15

So the following file is missing:
https://github.com/pandas-dev/pandas/blob/v0.23.4/pandas/tests/io/data/iris.csv

But is required:
https://github.com/pandas-dev/pandas/blob/v0.23.4/pandas/tests/io/test_sql.py#L259

Or at least that's what I think is happening.

[WillAyd]

Yea should have a skip for whenever the files are not bundled with the application. PRs certainly welcome

[TomAugspurger]

Can you also post how you're running pytest, the top of pytest --collect-only, and a snippet of some of the failing tests?

datapath skips the test when the --strict-data-files option is unset (the default).

[marianielias]

@TomAugspurger
Removing setup.cfg from distributions seems wrong, given that is used by versioneer.py.
Is not logical to force "--strict-data-files" when the data files aren't there...
Maybe there is a way to distribute a setup.cfg without the pytest hook ?

[TomAugspurger]

I don't think messing with setup.cfg just for the distribution is a good idea. It's probably best to flip the logic on `--strict-data-path` to be `--no-strict-data-path`. Then in the toplevel `pandas.test` we can add the `--no-strict-data-path` to the options.

…

On Thu, Sep 13, 2018 at 3:41 PM Elias M. Mariani ***@***.***> wrote: @TomAugspurger <https://github.com/TomAugspurger> Removing setup.cfg from distributions seems wrong, given that is used by versioneer.py. Is not logical to force "--strict-data-files" when the data files aren't there... Maybe there is a way to distribute a setup.cfg without the pytest hook ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22683 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIiS1t789Y2zJyLracA0M7YhguPxcks5uasLpgaJpZM4WmeoX> .

[marianielias]

@TomAugspurger
That is a project decision.
Even so there are 358 Failures because of "File Not Found" from several files, again, stripping the files like a butcher was gonna lead to something like this...

I leave you the test log without the --strict-data-files option.
test.log

[TomAugspurger]

stripping the files like a butcher was gonna lead to something like this...

? I spent a lot of time implementing datapath so that this would continue to work when we removed data files from the distributions.

I'll note that the documented way for users to run the tests is through pd.test(), which works correctly,

In [1]: import pandas as pd

In [2]: pd.test(['-k', 'test_pickle_v0_15_2'])
running: pytest -k test_pickle_v0_15_2 /Users/taugspurger/Envs/dask-dev/lib/python3.6/site-packages/pandas
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.6.5, pytest-3.7.2, py-1.5.3, pluggy-0.7.1
rootdir: /Users/taugspurger, inifile:
plugins: xdist-1.23.0, timeout-1.3.2, repeat-0.4.1, mock-1.10.0, forked-0.2, cov-2.5.1
collected 27542 items / 27540 deselected / 2 skipped

Envs/dask-dev/lib/python3.6/site-packages/pandas/tests/io/test_pickle.py s                                                                           [ 50%]
Envs/dask-dev/lib/python3.6/site-packages/pandas/tests/tseries/offsets/test_offsets.py s                                                             [100%]

======================================================= 4 skipped, 27540 deselected in 18.22 seconds =======================================================

if you want to use pytest from the pandas site-packages folder, then we can try to support that as well.

[marianielias]

@TomAugspurger
Sorry for the passive-aggressive comment, wasn't my intention to demerit your work.
Running pytest from the folder where the port is "built" is the standard on OpenBSD, and it would be more convenient if test could be performed that way, for now I can patch the port so it ignores the "--strict-data-files" option, but still gives lots of Errors of "file not found" if you could review that that would be great.
Apart from that, only one error in the hole test suite, so this will be available in the OpenBSD packages soon.

[mroeschke]

I think this has possibly been fixed in a more recent version of pandas so closing. Can reopen if seen again