Skip to content

bpo-23403: [lib2to3] Make grammar pickling faster #6491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 17, 2018
Merged

bpo-23403: [lib2to3] Make grammar pickling faster #6491

merged 1 commit into from
Apr 17, 2018

Conversation

ambv
Copy link
Contributor

@ambv ambv commented Apr 16, 2018 •
edited by bedevere-bot
Loading

  • Now uses pickle protocol 4

  • Doesn't wrap the grammar's __dict__ in ordered dictionaries anymore as
    dictionaries in Python 3.6+ are ordered by default

This still produces deterministic pickles (that hash the same with MD5).
Tested with different PYTHONHASHSEED values.

https://bugs.python.org/issue23403

@ambv
[lib2to3] Make grammar pickling faster
6a89c80 
* Now uses pickle protocol 4

* Doesn't wrap the grammar's `__dict__` in ordered dictionaries anymore as
dictionaries in Python 3.6+ are ordered by default

This still produces deterministic pickles (that hash the same with MD5).
Tested with different PYTHONHASHSEED values.
@ambv ambv requested a review from gpshead April 16, 2018 23:24
@ambv ambv changed the title [lib2to3] Make grammar pickling faster bpo-23403: [lib2to3] Make grammar pickling faster Apr 16, 2018
gpshead
gpshead approved these changes Apr 16, 2018
View reviewed changes
Copy link
Member

@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a very good idea. The grammar pickle is a cache.

I wish we'd generate it at CPython build time rather than ever do the runtime load and save a cache thing. But that's a separate feature request.

@ambv
Copy link
Contributor Author

ambv commented Apr 16, 2018 •
edited
Loading

The grammar pickle is a cache.

Yeah, and that cache is over twice as small with protocol 4 :)

I wish we'd generate it at CPython build time rather than ever do the runtime load and save a cache thing.

Yeah, that would make my life easier ;)

@ambv ambv merged commit 7661806 into python:master Apr 17, 2018
@bedevere-bot
Copy link

@ambv: Please replace # with GH- in the commit message next time. Thanks!

@ambv ambv deleted the fastgrammarpickle branch July 12, 2021 11:23
@serhiy-storchaka serhiy-storchaka mentioned this pull request May 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gpshead gpshead gpshead approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ambv @bedevere-bot @gpshead @the-knights-who-say-ni