Skip to content

Conversation

MarcoGorelli
Copy link
Member

@MarcoGorelli MarcoGorelli commented Dec 31, 2022

So, #40906 consolidated a bunch of custom checks into a flake8 extension

As it stands, flake8 is the slowest pre-commit hook we have (even without the extension - see timings)

In #50160, Joris tried adding ruff, which would more than an order of magnitude faster (!), and has more checks. Several projects are moving towards it

So, I suggest we revert #40906 , so that we can rip out flake8 and can replace it with ruff without losing the custom checks

This is mostly a pure revert of #40906, with some adjustments for things which have since changed

I removed the import pandas.core.common as com one, as that one was noqa'd in a few places anyway (if the check isn't a flake8 extension, then it's not so easy to turn off the check, we it's done for a whole file) and doesn't seem too important anyway


Overall, I think that this + the ruff PR would be a big improvement to developer workflow. It would mean more hooks (for now, maybe then can be consolidated anyway), but they would run a lot faster. And I'd expect the speed of the checks to matter much more to developer workflow than the speed of them

@MarcoGorelli MarcoGorelli force-pushed the rip-out-pandas-dev-flaker-from-flake8 branch from 2357226 to a747171 Compare December 31, 2022 21:33
@MarcoGorelli MarcoGorelli marked this pull request as ready for review January 1, 2023 08:13
@MarcoGorelli MarcoGorelli added the Code Style Code style, linting, code_checks label Jan 1, 2023
pytest.xfail("known failure on some windows platforms")
pytest.mark.xfail("known failure on some windows platforms")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the pytest.xfail check in pandas-dev-flaker only checks decorators. My bad here, a simple regex ended up being the better solution here

Copy link
Member

@mroeschke mroeschke Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will make it appear that this test will pass when it hits this line as this mark is not attached to anything.

Probably needs something like

except OverflowError:
     if is_platform_windows():
         request.node.add_marker(pytest.mark.xfail(reason=...)
     raise

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, thanks!

pytest.xfail("known failure on some windows platforms")
request.node.add_marker(
pytest.mark.xfail("known failure on some windows platforms")
)
else:
Copy link
Member

@mroeschke mroeschke Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think this else should be removed now too so the xfail will be recognized after the raise

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending green

@MarcoGorelli MarcoGorelli added this to the 2.0 milestone Jan 5, 2023
@MarcoGorelli
Copy link
Member Author

thanks - merging to clear the queue then, the only failures are the sqlalchemy ones, which hopefully #50580 will address

@MarcoGorelli MarcoGorelli merged commit 3a0db10 into pandas-dev:main Jan 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code Style Code style, linting, code_checks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants