Use custom tokenization function for NetCDFDataProxy objects #6231
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bouweandela
added
the
benchmark_this
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6231 +/- ##
=======================================
Coverage 89.88% 89.88%
=======================================
Files 88 88
Lines 23430 23432 +2
Branches 4361 4361
=======================================
+ Hits 21059 21061 +2
Misses 1644 1644
Partials 727 727 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
⏱️ Performance Benchmark Report: 3fc5893Performance shiftsFull benchmark resultsGenerated by GHA run |
Member
Looks really neat + I don't doubt you're right this is worth it for some cases. |
bouweandela
added
benchmark_this
Contributor
⏱️ Performance Benchmark Report: 23e9462Performance shiftsFull benchmark resultsGenerated by GHA run |
Member
Author
|
The benchmark I added just shows: On my laptop the timings are a bit more positive, but maybe this isn't really worth the effort. Shall I just close this? |
Member
Author
|
As discussed at the peloton meeting, this is probably useful after all. The benchmark is likely slower in CI than on my laptop because it has different hardware and therefore spends more time on I/O, reducing the relative speedup of this change on that particular machine. |
bouweandela
added
benchmark_this
bouweandela
added
benchmark_this
This comment was marked as outdated.
This comment was marked as outdated.
Contributor
⏱️ Performance Benchmark Report: a09fe71Performance shiftsFull benchmark resultsGenerated by GHA run |
Member
Author
|
The improvement is still ~50 ms, but since the overall time is now shorter the benefit becomes relatively larger. CI results: and on my laptop |
bouweandela
added
benchmark_this
Member
Author
|
Benchmark result from my laptop with the most recent commit ( (changed because initially I forgot to fetch |
Contributor
⏱️ Performance Benchmark Report: 0cd2656Performance shiftsFull benchmark resultsGenerated by GHA run |
Member
Author
|
It looks like the advantage is rather minimal for the benchmarked case now that we are creating fewer arrays since #6252. Let's just close this. |
Contributor
Sincere apologies for not getting round to looking at this in good time @bouweandela. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🚀 Pull Request
Description
Use custom tokenization function for NetCDFDataProxy objects. This speeds up loading small NetCDF files. With the multiple_cubes.nc file from #6223, the speedup is 10 to 20 %.
Consult Iris pull request check list
Add any of the below labels to trigger actions on this PR: