Skip to content

Conversation

DaniPopes
Copy link
Contributor

criterion's Bencher implements Send + Sync, which allows running the benchmark itself on another thread. This allows something like |b| /* my rayon thread pool */.install(|| b.iter(|| /* bench */)).

Copy link

codspeed-hq bot commented Aug 2, 2025

CodSpeed Instrumentation Performance Report

Merging #116 will degrade performances by 15.89%

Comparing DaniPopes:criterion-bencher-send (97babdc) with main (43a2ff0)

Summary

⚡ 8 improvements
❌ 10 regressions
✅ 149 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
Iterative 162.5 ns 132.5 ns +22.64%
Iterative[20] 162.5 ns 132.5 ns +22.64%
Iterative[21] 164.2 ns 134.2 ns +22.36%
fibonacci_custom_measurement 916.7 ns 945.8 ns -3.08%
sleep_100ms 595.3 ns 536.9 ns +10.86%
sleep_10ms 566.1 ns 536.9 ns +5.43%
sleep_1ms 566.1 ns 536.9 ns +5.43%
sleep_50ms 595.3 ns 536.9 ns +10.86%
from_elem[1024] 1.8 µs 1.9 µs -3.1%
from_elem_decimal[1024] 1.8 µs 1.9 µs -3.1%
bench_array1[10] 225 ns 254.2 ns -11.48%
bench_array1[4] 154.4 ns 183.6 ns -15.89%
bench_array2[10] 225 ns 254.2 ns -11.48%
bench_array2[4] 154.4 ns 183.6 ns -15.89%
init_array[4] 154.4 ns 183.6 ns -15.89%
recursive[0] 211.1 ns 240.3 ns -12.14%
recursive[5] 291.1 ns 320.3 ns -9.11%
recursive_memoized[BTreeMap<u64, u64>, 0] 430 ns 400.8 ns +7.28%

@art049 art049 requested a review from not-matthias August 2, 2025 15:28
Copy link

codspeed-hq bot commented Aug 2, 2025

CodSpeed WallTime Performance Report

Merging #116 will degrade performances by 20.4%

Comparing DaniPopes:criterion-bencher-send (97babdc) with main (43a2ff0)

Summary

⚡ 15 improvements
❌ 8 regressions
✅ 131 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
build_vec 214 ns 222 ns -3.6%
large_drop 168.4 µs 161.5 µs +4.33%
large_setup 16 ns 12 ns +33.33%
from_elem[2048] 255 ns 271 ns -5.9%
from_elem[4096] 347 ns 334 ns +3.89%
generate_combinations[6] 1.4 µs 1.3 µs +7.28%
generate_combinations[7] 2.1 µs 2 µs +5.96%
generate_parentheses[6] 68.3 µs 72.2 µs -5.5%
graph_coloring[3] 1.2 µs 1.1 µs +4.35%
graph_coloring[4] 962 ns 896 ns +7.37%
graph_coloring[5] 971 ns 1,025 ns -5.27%
hamiltonian_cycle[4] 575 ns 540 ns +6.48%
hamiltonian_cycle[5] 606 ns 656 ns -7.62%
hamiltonian_cycle[7] 899 ns 776 ns +15.85%
n_queens_solver[5] 9.1 µs 7.5 µs +21.96%
permutations[5] 11.6 µs 14.6 µs -20.4%
permutations[6] 76.4 µs 68.8 µs +11.01%
permutations[7] 792.1 µs 767 µs +3.27%
rat_in_maze[5] 573 ns 603 ns -4.98%
rat_in_maze[8] 902 ns 932 ns -3.22%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Copy link
Member

@not-matthias not-matthias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Minor style nitpick: You could remove the let codspeed = &mut *self.codspeed; variables altogether and use it directly when calling the functions (e.g. self.codspeed.start_benchmark)

@DaniPopes DaniPopes force-pushed the criterion-bencher-send branch from ad60ffa to 1f8a754 Compare August 7, 2025 21:24
@DaniPopes DaniPopes requested a review from not-matthias August 7, 2025 21:29
@DaniPopes
Copy link
Contributor Author

Any chance this can get merged and released soon? Thanks! :D @art049 @not-matthias

`criterion`'s `Bencher` implements `Send + Sync`, which allows running
the benchmark itself on another thread. This allows something like
`|b| /* my rayon thread pool */.install(|| b.iter(|| /* bench */))`.
@not-matthias not-matthias merged commit 97babdc into CodSpeedHQ:main Sep 26, 2025
13 of 14 checks passed
@DaniPopes DaniPopes deleted the criterion-bencher-send branch September 26, 2025 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants