-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 17.78 μs | 13.93 μs | 0.78 | 0.07 | False | ||||
| 16.69 μs | 13.02 μs | 0.78 | 0.18 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'Payloads
System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.926358534046983 < 16.895757058597457.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/9/2023 2:21:24 AM, 6/13/2023 11:57:38 AM, 6/26/2023 10:42:28 PM, 7/14/2023 2:26:40 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 13.250719592916491 (T) = (0 -13913.871853855888) / Math.Sqrt((3824629.730084603 / (299)) + (1227.5601673763833 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09753578568802929 = (15417.643861328816 - 13913.871853855888) / 15417.643861328816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<Int32>.Queue(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.02044620572855 < 15.965795082085254.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 6/14/2023 4:56:58 PM, 6/27/2023 10:31:09 PM, 7/12/2023 2:12:47 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 13.518558055440625 (T) = (0 -12823.361044844341) / Math.Sqrt((3398616.8340120437 / (299)) + (41642.88185589672 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.11140396011249862 = (14431.035554094764 - 12823.361044844341) / 14431.035554094764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 15.29 ns | 12.14 ns | 0.79 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'Payloads
System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.138212927747945 < 14.12461957880728.
IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/26/2023 10:42:28 PM, 7/17/2023 10:23:26 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 10.477843174579778 (T) = (0 -12.17519336565023) / Math.Sqrt((1.7095913731656405 / (299)) + (0.027364563015938776 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.06861212603954862 = (13.072097786584683 - 12.17519336565023) / 13.072097786584683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 12.19 ns | 9.82 ns | 0.81 | 0.13 | False | ||||
| 8.04 ns | 6.22 ns | 0.77 | 0.16 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'Payloads
System.Memory.Span<Byte>.BinarySearch(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.81615118567626 < 11.448077182002013.
IsChangePoint: Marked as a change because one of 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 15.803529260116846 (T) = (0 -9.807756634444306) / Math.Sqrt((1.0322223659369698 / (299)) + (0.001739450748126397 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.08780097495747097 = (10.751772765802993 - 9.807756634444306) / 10.751772765802993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.BinarySearch(Size: 33)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.215500259587033 < 7.392055534857006.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 17.373366484796332 (T) = (0 -6.197489973090694) / Math.Sqrt((0.4997593162439849 / (299)) + (0.0038071147851697954 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.10952936787368991 = (6.959791541122495 - 6.197489973090694) / 6.959791541122495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in PerfLabTests.CastingPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 306.35 μs | 278.43 μs | 0.91 | 0.05 | False | ||||
| 306.50 μs | 278.52 μs | 0.91 | 0.05 | False | ||||
| 306.27 μs | 278.52 μs | 0.91 | 0.05 | False | ||||
| 306.47 μs | 278.54 μs | 0.91 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'Payloads
PerfLabTests.CastingPerf.FooObjIsFoo
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 278.43404220779223 < 291.06085734577914.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 6/13/2023 11:57:38 AM, 6/19/2023 3:58:06 AM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.476460690713015 (T) = (0 -278567.9702831205) / Math.Sqrt((191291721.33953077 / (299)) + (41837.321778758596 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.053063733513141594 = (294178.1618699747 - 278567.9702831205) / 294178.1618699747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.ObjrefValueTypeObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 278.51632305194806 < 291.0379301136363.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.740393965711085 (T) = (0 -278523.90359338623) / Math.Sqrt((194709653.75130966 / (299)) + (22679.605483877862 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.05414941488679291 = (294469.2406782729 - 278523.90359338623) / 294469.2406782729 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.IntObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 278.5159665991903 < 290.9673777056277.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.96656144380105 (T) = (0 -278543.40251649974) / Math.Sqrt((190173036.8564499 / (299)) + (8333.203446077609 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05410004263883023 = (294474.4846945209 - 278543.40251649974) / 294474.4846945209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf.ScalarValueTypeObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 278.5428733766234 < 291.02528169642864.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.854873765137754 (T) = (0 -278499.0320901078) / Math.Sqrt((190987799.37767142 / (299)) + (4195.694009057251 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.053916225299831394 = (294370.3713536037 - 278499.0320901078) / 294370.3713536037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 6.98 ns | 5.95 ns | 0.85 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'Payloads
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetArray
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.953102439839692 < 6.626915353075916.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 9.234122510916878 (T) = (0 -6.107214472951942) / Math.Sqrt((0.1555222312603314 / (299)) + (0.016720816518866675 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.05442446792345568 = (6.45872726797415 - 6.107214472951942) / 6.45872726797415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 21.25 μs | 17.50 μs | 0.82 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'Payloads
System.Collections.ContainsFalse<Int32>.Array(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.495289196495154 < 20.185357586657283.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.77445763015441 (T) = (0 -17446.482646709967) / Math.Sqrt((3151846.2810140047 / (299)) + (6155.178766437725 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.10603663619974112 = (19515.88102284703 - 17446.482646709967) / 19515.88102284703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Char
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 24.86 ns | 16.52 ns | 0.66 | 0.28 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'Payloads
System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.518166316361633 < 23.5014757046086.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 6/13/2023 11:57:38 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 6.930619831799248 (T) = (0 -17.142065332390054) / Math.Sqrt((17.26891698841828 / (299)) + (4.236725006734533 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.17958567703068168 = (20.894400368764806 - 17.142065332390054) / 20.894400368764806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
| Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in PerfLabTests.LowLevelPerf
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 83.51 μs | 69.62 μs | 0.83 | 0.06 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'Payloads
PerfLabTests.LowLevelPerf.ObjectStringIsString
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.61505769230769 < 79.3673947726697.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/11/2023 9:15:05 PM, 6/26/2023 10:42:28 PM, 7/14/2023 2:26:40 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 19.81377176061112 (T) = (0 -69633.79349435287) / Math.Sqrt((47706114.621664554 / (299)) + (292.3679085638735 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10206319742696121 = (77548.65742757972 - 69633.79349435287) / 77548.65742757972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository












