bpo-46576: Speed up test_peg_generator by using a static library for shared sources #32338
Conversation
|
Ah, fantastic idea! Let's check that the CI works and I can land it afterwards. Thanks for working on this @jkloth 🙏 |
gpshead
left a comment
gpshead
left a comment
There was a problem hiding this comment.
Great change! My comment are all around expanding on some code comments. otherwise this is good.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
|
Hrm, in my zeal over the improvement on Windows, I did forget to run through a POSIX build. Once I get that sorted out, I'll address the other changes. |
|
I have resolved the issues when running on POSIX (well, Linux). I am quite surprised by the speedup there, before 33s, after 6s. |
gpshead
added
performance
|
LGTM once it proves it self with testing on a variety of platforms. But I'd wait until after Pablo has released the Alpha and unblocked main before applying the test-with-buildbots label to better verify this change (to avoid tying up buildbot resources until after the release) |
|
One last change for MacOS (due to deprecated linker option). Hopefully this run (of test_peg_generator) should be error/warning free. And, yes, a run through the buildbots would be very beneficial. |
|
Hoorah, the MacOS run of test_peg_generator is clean! It is a shame that network errors cause the GHA to fail. Is there something that can be done on our end to, say, add retries to parts of the scripts that can fail on network errors? |
gpshead
added
the
🔨 test-with-buildbots
bedevere-bot
removed
the
🔨 test-with-buildbots
|
Looks like everything passed. The failures are refleaks and 1 slow SSL handshake (gentoo). |
|
Sorry, @jkloth and @gpshead, I could not cleanly backport this to |
…ry for shared sources (pythonGH-32338) Speed up test_peg_generator by using a static library for shared sources to avoid recompiling as much code.. (cherry picked from commit 612e422) Co-authored-by: Jeremy Kloth <[email protected]>
|
Well, I see that you removed the 3.10 backport label while I was working on the backport. Is it desired, or should I just close it? |
6 participants
By using a static library for the shared (unchanging) sources between extensions, this reduces the number of spawned processes for compiling thus reducing the runtime ~25%.
On my machine: before 85s, after 65s.
https://bugs.python.org/issue46576
Automerge-Triggered-By: GH:pablogsal