[Perf] Linux/arm64: 5 Regressions on 9/27/2022 12:44:07 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	f8b032cd4fd5f544211d37ac25816cbd4deb67b7
Compare	7fc8fb56327e9696ce9baa80f997e3f50675af1f
Diff	Diff

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CopyTo - Duration of single invocation	24.92 μs	35.59 μs	1.43	0.51	False
CopyTo - Duration of single invocation	25.38 μs	34.41 μs	1.36	0.50	False
ReadAllBytes - Duration of single invocation	6.31 μs	9.05 μs	1.43	0.56	False
ReadAllBytes - Duration of single invocation	4.39 μs	6.72 μs	1.53	0.46	False
ReadAllBytes - Duration of single invocation	5.04 μs	8.14 μs	1.62	0.56	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.CopyTo(size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.594804657061694 > 26.09059472858944.
IsChangePoint: Marked as a change because one of 8/23/2022 6:13:58 PM, 9/26/2022 7:45:50 PM, 9/29/2022 12:14:35 AM falls between 9/20/2022 9:08:11 AM and 9/29/2022 12:14:35 AM.
IsRegressionStdDev: Marked as regression because -13.613009703729203 (T) = (0 -36511.77374580777) / Math.Sqrt((1566868.0637298285 / (42)) + (5175036.831680358 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.44591350254795964 = (25251.69982953161 - 36511.77374580777) / 25251.69982953161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.CopyTo(size: 4096)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.40838088609899 > 26.208162399025255.
IsChangePoint: Marked as a change because one of 8/23/2022 11:36:32 PM, 9/26/2022 7:45:50 PM, 9/29/2022 12:14:35 AM falls between 9/20/2022 9:08:11 AM and 9/29/2022 12:14:35 AM.
IsRegressionStdDev: Marked as regression because -15.495849004882762 (T) = (0 -35435.653598683675) / Math.Sqrt((872448.9772897566 / (42)) + (3165962.1549307415 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.3931895323425536 = (25434.91231885803 - 35435.653598683675) / 25434.91231885803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 16384)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.045063286511875 > 6.659189162809567.
IsChangePoint: Marked as a change because one of 8/23/2022 11:36:32 PM, 9/26/2022 7:45:50 PM, 9/29/2022 12:14:35 AM falls between 9/20/2022 9:08:11 AM and 9/29/2022 12:14:35 AM.
IsRegressionStdDev: Marked as regression because -13.899161751698783 (T) = (0 -9163.779053103408) / Math.Sqrt((65225.11082022433 / (42)) + (304692.97658290406 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.4326243571176506 = (6396.498152202543 - 9163.779053103408) / 6396.498152202543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.71578038848459 > 4.6212490147787175.
IsChangePoint: Marked as a change because one of 8/23/2022 11:36:32 PM, 9/26/2022 7:45:50 PM, 9/29/2022 12:14:35 AM falls between 9/20/2022 9:08:11 AM and 9/29/2022 12:14:35 AM.
IsRegressionStdDev: Marked as regression because -11.115603545421076 (T) = (0 -7349.238629170683) / Math.Sqrt((14760.799831971182 / (42)) + (538954.3655574843 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.6490678119208728 = (4456.601830466946 - 7349.238629170683) / 4456.601830466946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 4096)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.144012421752738 > 5.22310083708533.
IsChangePoint: Marked as a change because one of 8/23/2022 11:36:32 PM, 9/26/2022 7:45:50 PM, 9/29/2022 12:14:35 AM falls between 9/20/2022 9:08:11 AM and 9/29/2022 12:14:35 AM.
IsRegressionStdDev: Marked as regression because -15.336957768962828 (T) = (0 -7967.533070545622) / Math.Sqrt((71696.64768731919 / (42)) + (282607.2829072366 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.5883932274721652 = (5016.096098083649 - 7967.533070545622) / 5016.096098083649 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 18.04
Baseline	f8b032cd4fd5f544211d37ac25816cbd4deb67b7
Compare	7fc8fb56327e9696ce9baa80f997e3f50675af1f
Diff	Diff

Regressions in System.IO.Tests.Perf_File

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadAllBytes - Duration of single invocation	10.42 μs	17.88 μs	1.72	0.70	False
ReadAllBytesAsync - Duration of single invocation	20.04 μs	25.64 μs	1.28	0.42	False
ReadAllBytes - Duration of single invocation	14.78 μs	18.02 μs	1.22	0.62	False
CopyTo - Duration of single invocation	57.58 μs	84.33 μs	1.46	0.67	False
CopyTo - Duration of single invocation	66.96 μs	82.26 μs	1.23	0.70	False
ReadAllBytesAsync - Duration of single invocation	19.40 μs	26.50 μs	1.37	0.48	False
ReadAllBytesAsync - Duration of single invocation	21.99 μs	28.13 μs	1.28	0.33	False
ReadAllBytes - Duration of single invocation	10.28 μs	18.62 μs	1.81	0.62	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.ReadAllBytes(size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.880258174400716 > 11.367498845718236.
IsChangePoint: Marked as a change because one of 8/23/2022 11:36:32 PM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -15.843873059077056 (T) = (0 -17387.076018522013) / Math.Sqrt((925453.707812449 / (46)) + (452906.4910669946 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.6049183154373124 = (10833.620534627855 - 17387.076018522013) / 10833.620534627855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.637917568280777 > 19.915621227862665.
IsChangePoint: Marked as a change because one of 8/24/2022 9:15:22 AM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -7.768856975332395 (T) = (0 -24264.91878443035) / Math.Sqrt((703985.7805518269 / (46)) + (1421729.9572451683 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.28855678921472316 = (18831.082174669198 - 24264.91878443035) / 18831.082174669198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 16384)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.0175585143364 > 13.734329323257535.
IsChangePoint: Marked as a change because one of 8/24/2022 2:39:26 AM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -45.678571023504205 (T) = (0 -18054.72670233734) / Math.Sqrt((506719.2192330659 / (46)) + (1186.585818007431 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.37035667906065795 = (13175.20246970545 - 18054.72670233734) / 13175.20246970545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.CopyTo(size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.33023530061892 > 69.78085550978597.
IsChangePoint: Marked as a change because one of 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -12.54604967935994 (T) = (0 -83400.07023382669) / Math.Sqrt((35589961.843964495 / (46)) + (6234768.902556793 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.3405644626890926 = (62212.65187541307 - 83400.07023382669) / 62212.65187541307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.CopyTo(size: 4096)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.25625062321936 > 60.803178763600705.
IsChangePoint: Marked as a change because one of 8/22/2022 2:51:08 PM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -12.098185039835984 (T) = (0 -81516.04289418201) / Math.Sqrt((46450472.8239382 / (46)) + (4541878.275959134 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.30851771400694333 = (62296.47640348984 - 81516.04289418201) / 62296.47640348984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 4096)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.49839954080904 > 20.92181218847646.
IsChangePoint: Marked as a change because one of 8/20/2022 9:50:55 AM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -5.048102610963182 (T) = (0 -25852.130816232147) / Math.Sqrt((755069.1869547741 / (46)) + (4314456.543117648 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.3080516570122471 = (19763.845470202326 - 25852.130816232147) / 19763.845470202326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 16384)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.128809680451134 > 23.600294132606976.
IsChangePoint: Marked as a change because one of 8/24/2022 9:42:22 AM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -9.834963570681262 (T) = (0 -27340.552830874833) / Math.Sqrt((402061.6387114011 / (46)) + (714672.5417077849 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.21767847762798037 = (22453.01475980246 - 27340.552830874833) / 22453.01475980246 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllBytes(size: 4096)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.61583801708633 > 11.654701650660035.
IsChangePoint: Marked as a change because one of 9/8/2022 10:00:03 PM, 9/26/2022 7:45:50 PM, 9/27/2022 6:39:02 PM falls between 9/19/2022 6:07:30 AM and 9/27/2022 6:39:02 PM.
IsRegressionStdDev: Marked as regression because -25.23918356318042 (T) = (0 -18650.311264069078) / Math.Sqrt((1127050.2284161362 / (46)) + (161057.2908119887 / (3))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (3) - 2, .025) and -0.6087628022144346 = (11592.952819643295 - 18650.311264069078) / 11592.952819643295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository