- 
                Notifications
    You must be signed in to change notification settings 
- Fork 5.2k
Closed
Labels
area-System.Numericstenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Description
Run Information
| Architecture | x64 | 
|---|---|
| OS | ubuntu 18.04 | 
| Baseline | 41f57b7862dca221687a6cb638c35e33ee8e25db | 
| Compare | 09613f3ed6cb5ce62e955d2a1979115879d707bb | 
| Diff | Diff | 
Regressions in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL | 
|---|---|---|---|---|---|---|---|---|---|---|
| Array - Duration of single invocation | 269.70 μs | 291.19 μs | 1.08 | 0.15 | False | 
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'Payloads
Histogram
System.Collections.ContainsTrue<String>.Array(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 291.1920761574074 > 283.4228556552553.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 10:51:47 AM, 10/25/2022 2:34:53 PM, 12/1/2022 11:48:13 PM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -13.679745201453423 (T) = (0 -294301.5516878831) / Math.Sqrt((34689258.78466946 / (32)) + (26346676.045331515 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.08013880888349276 = (272466.41752655274 - 294301.5516878831) / 272466.41752655274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 | 
|---|---|
| OS | ubuntu 18.04 | 
| Baseline | 41f57b7862dca221687a6cb638c35e33ee8e25db | 
| Compare | 09613f3ed6cb5ce62e955d2a1979115879d707bb | 
| Diff | Diff | 
Regressions in System.Tests.Perf_UInt64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL | 
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 1.20 ns | 2.40 ns | 2.01 | 0.21 | False | 
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'Payloads
Histogram
System.Tests.Perf_UInt64.ToString(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4033548563465605 > 1.4579393401132879.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 12/1/2022 11:48:13 PM, 12/6/2022 5:44:36 AM falls between 11/27/2022 4:37:47 PM and 12/6/2022 5:44:36 AM.
IsRegressionStdDev: Marked as regression because -20.882326878308504 (T) = (0 -2.6873065212221836) / Math.Sqrt((0.013914976613451827 / (32)) + (0.06680088451249501 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -1.0014129777233116 = (1.342704654728033 - 2.6873065212221836) / 1.342704654728033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
area-System.Numericstenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
_1.png)
