MAINT: Add more pytest idiom to parser tests #23712

gfyoung · 2018-11-15T01:09:02Z

This is only a start. To keep the diff small, we are only converting c_parser_only.

If this looks good, the other test classes will soon follow in subsequent PR's.

pep8speaks · 2018-11-15T01:09:05Z

Hello @gfyoung! Thanks for submitting the PR.

There are no PEP8 issues in the file pandas/tests/io/parser/conftest.py !
There are no PEP8 issues in the file pandas/tests/io/parser/test_c_parser_only.py !
There are no PEP8 issues in the file pandas/tests/io/parser/test_parsers.py !

This is only a start. To keep the diff small, we are only converting c_parser_only. The other test classes will soon follow.

codecov · 2018-11-15T06:49:39Z

Codecov Report

Merging #23712 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #23712   +/-   ##
=======================================
  Coverage   92.24%   92.24%           
=======================================
  Files         161      161           
  Lines       51336    51336           
=======================================
  Hits        47357    47357           
  Misses       3979     3979

Flag	Coverage Δ
#multiple	`90.64% <ø> (ø)`	⬆️
#single	`42.34% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6920363...d916288. Read the comment docs.

jreback · 2018-11-15T13:21:05Z

how do you run all of the tests on the c parser though? don't you need to also inject the fixture into the current test_parser.py?

pandas/tests/io/parser/conftest.py

gfyoung · 2018-11-15T18:02:35Z

@jreback @jbrockmendel : Here is the plan:

c_parser_only.py is now separate from test_parsers.py, so you can just run pytest test_c_parser_only.py (the fixtures get automatically injected per the newly created conftest.py). The new format of those tests is 100% functional (no classes), which is more pytest-idiomatic.
If this format looks good, I will convert all other files (e.g. header.py, comment.py, mangle_dupe_cols.py similarly in subsequent PR's (we will be in this weird intermediate state with test_parsers.py and conftest.py co-existing).

Once that is done, we no longer need test_parsers.py. All tests will be nice and modular and not all clumped into one giant test class as they are currently.

Hopefully that clears things up?

gfyoung · 2018-11-17T22:40:44Z

@jreback : Any further thoughts on this?

jreback · 2018-11-17T22:48:40Z

lgtm @gfyoung keep em coming!

Adds more idiom to the following: * parser/comment --> parser/test_comment * parser/converters --> parser/test_converters * parser/compression --> parser/test_compression Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/comment --> parser/test_comment * parser/converters --> parser/test_converters * parser/compression --> parser/test_compression * parser/dialect --> parser/test_dialect Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/comment --> parser/test_comment * parser/converters --> parser/test_converters * parser/compression --> parser/test_compression * parser/dialect --> parser/test_dialect Builds off of gh-23712.

This is only a start. To keep the diff small, we are only converting c_parser_only. The other test classes will soon follow.

Adds more idiom to the following: * parser/comment --> parser/test_comment * parser/converters --> parser/test_converters * parser/compression --> parser/test_compression * parser/dialect --> parser/test_dialect Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/header --> parser/test_header * parser/dtypes --> parser/test_dtypes * parser/index_col --> parser/test_index_col * parser/mangle_dupes --> parser/test_mangle_dupes Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/header --> parser/test_header * parser/dtypes --> parser/test_dtypes * parser/index_col --> parser/test_index_col * parser/mangle_dupes --> parser/test_mangle_dupes Builds off of gh-23712.

Adds more idiom to the following: * parser/na_values --> parser/test_na_values * parser/parse_dates --> parser/test_parse_dates * parser/multithread --> parser/test_multi_thread Also adds new test suite: test_date_converters Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/skiprows --> parser/test_skiprows * parser/na_values --> parser/test_na_values * parser/parse_dates --> parser/test_parse_dates * parser/multithread --> parser/test_multi_thread Also adds new test suite: test_date_converters Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/skiprows --> parser/test_skiprows * parser/na_values --> parser/test_na_values * parser/parse_dates --> parser/test_parse_dates * parser/multithread --> parser/test_multi_thread Also adds new test suite: test_date_converters Builds off of gh-23712.

Adds more idiom to the following: * parser/common --> parser/test_common * parser/quoting --> parser/test_quoting * parser/usecols --> parser/test_usecols * parser/python_parser_only --> parser/test_python_parser_only Also deletes parser/test_parsers. Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/common --> parser/test_common * parser/quoting --> parser/test_quoting * parser/usecols --> parser/test_usecols * parser/python_parser_only --> parser/test_python_parser_only Also: * Finally delete parser/test_parsers! * Bug in capture_stdout decorator in which we were forgetting to use compat.wraps. Builds off of pandas-devgh-23712.

Adds more idiom to the following: * parser/common --> parser/test_common * parser/quoting --> parser/test_quoting * parser/usecols --> parser/test_usecols * parser/python_parser_only --> parser/test_python_parser_only Also: * Finally delete parser/test_parsers! * Bug in capture_stdout decorator in which we were forgetting to use compat.wraps. Builds off of gh-23712.