[Perf] Linux/arm64: 3189 Improvements on 5/11/2023 10:01:37 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BuildProvider - Duration of single invocation	9.41 μs	5.13 μs	0.54	0.08	True
BuildProvider - Duration of single invocation	9.84 μs	4.81 μs	0.49	0.08	True
BuildProvider - Duration of single invocation	9.91 μs	5.89 μs	0.59	0.06	True
Singleton - Duration of single invocation	18.25 μs	6.57 μs	0.36	0.14	True
Scoped - Duration of single invocation	19.02 μs	7.01 μs	0.37	0.12	True
Scoped - Duration of single invocation	18.54 μs	9.37 μs	0.51	0.12	True
Singleton - Duration of single invocation	17.87 μs	7.27 μs	0.41	0.13	True
Scoped - Duration of single invocation	18.52 μs	7.58 μs	0.41	0.14	True
BuildProvider - Duration of single invocation	9.61 μs	4.26 μs	0.44	0.09	True
Singleton - Duration of single invocation	17.76 μs	6.50 μs	0.37	0.14	True
Transient - Duration of single invocation	17.89 μs	7.10 μs	0.40	0.13	True
Transient - Duration of single invocation	17.29 μs	8.30 μs	0.48	0.15	True
Scoped - Duration of single invocation	18.75 μs	6.97 μs	0.37	0.13	True
Singleton - Duration of single invocation	17.50 μs	6.66 μs	0.38	0.14	True
Transient - Duration of single invocation	17.67 μs	6.61 μs	0.37	0.12	True
Transient - Duration of single invocation	17.55 μs	7.17 μs	0.41	0.13	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.DependencyInjection.TimeToFirstService* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Microsoft.Extensions.DependencyInjection.TimeToFirstService* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.127124027728672 < 9.139003631111388.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 24.067030248399092 (T) = (0 -4715.943197824442) / Math.Sqrt((28074.239793403543 / (299)) + (439878.290017125 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5053723704524227 = (9534.330304471647 - 4715.943197824442) / 9534.330304471647 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.805112219081104 < 9.241212708517795.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 19.331679906931893 (T) = (0 -5103.021594338448) / Math.Sqrt((36756.631279395566 / (299)) + (581832.1602469835 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.46588777715151725 = (9554.212347216919 - 5103.021594338448) / 9554.212347216919 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.886936033863034 < 9.160555879145116.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 21.984200302128624 (T) = (0 -4777.012057236928) / Math.Sqrt((37771.27239476612 / (299)) + (510570.2717085535 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4982016565636832 = (9519.784430781363 - 4777.012057236928) / 9519.784430781363 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.569137866164505 < 17.076414736347992.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 49.994119048307695 (T) = (0 -7048.634673766786) / Math.Sqrt((151667.8279951913 / (299)) + (501845.80048449023 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6037036573131221 = (17786.27232837236 - 7048.634673766786) / 17786.27232837236 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.0144439853709875 < 18.002945178156285.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 41.227487170964245 (T) = (0 -7907.892892920066) / Math.Sqrt((152229.94752240437 / (299)) + (755916.222117595 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5783673011885043 = (18755.407052657323 - 7907.892892920066) / 18755.407052657323 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.371423592660506 < 17.877619584540497.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 38.47686685068837 (T) = (0 -7776.5812545682475) / Math.Sqrt((176254.67702283015 / (299)) + (891968.720560981 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5857570100002273 = (18772.994214271468 - 7776.5812545682475) / 18772.994214271468 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.267801471352489 < 17.166280817495306.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 121.63317452980283 (T) = (0 -6695.823367758153) / Math.Sqrt((157824.91342215473 / (299)) + (85484.60830366155 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6233345414057866 = (17776.579229611943 - 6695.823367758153) / 17776.579229611943 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.577581965582843 < 18.02525782537563.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 57.33042416863501 (T) = (0 -7574.126805792519) / Math.Sqrt((165860.80016994925 / (299)) + (408522.3765124775 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5950675046260667 = (18704.665326496513 - 7574.126805792519) / 18704.665326496513 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.259263592394052 < 9.224035159881971.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 28.833995868718596 (T) = (0 -4684.6048931417145) / Math.Sqrt((27100.43365473413 / (299)) + (311793.1012204832 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5093025364783993 = (9546.829240814888 - 4684.6048931417145) / 9546.829240814888 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.498996144818161 < 17.064638597208216.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 42.87407806834531 (T) = (0 -7131.879929416961) / Math.Sqrt((147725.85874358955 / (299)) + (675663.7525252993 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5993411659212864 = (17800.38107936946 - 7131.879929416961) / 17800.38107936946 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.104385488691764 < 17.016984894570825.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 45.30631485154616 (T) = (0 -7155.563156014872) / Math.Sqrt((147481.35594783357 / (299)) + (582090.0682201791 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5940377277863457 = (17626.177715965103 - 7155.563156014872) / 17626.177715965103 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.300510342486172 < 16.698346869581773.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 57.06160449572939 (T) = (0 -6966.241010423683) / Math.Sqrt((333415.2765205223 / (299)) + (370839.2551245695 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6045321724448381 = (17615.19022543496 - 6966.241010423683) / 17615.19022543496 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.972860868044098 < 18.349149728675087.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 56.024245203324114 (T) = (0 -7538.7782595894905) / Math.Sqrt((207013.49280486893 / (299)) + (430883.8410052488 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5973844466823269 = (18724.508274624994 - 7538.7782595894905) / 18724.508274624994 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.6601088279967415 < 16.74996731495321.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 41.14823719142691 (T) = (0 -7380.263854912736) / Math.Sqrt((171860.46014777137 / (299)) + (702011.1986703275 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5858909084564001 = (17822.028073333662 - 7380.263854912736) / 17822.028073333662 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.6136695656105875 < 16.69877977190778.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 37.88542176256016 (T) = (0 -7411.954927321401) / Math.Sqrt((145799.59031580016 / (299)) + (787937.0398549394 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5785315561241486 = (17586.025798659037 - 7411.954927321401) / 17586.025798659037 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.169362396541237 < 17.205802428232328.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 54.57589706645656 (T) = (0 -7195.200750548023) / Math.Sqrt((163020.3710153395 / (299)) + (395727.80011515936 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5917578389565222 = (17624.835054167113 - 7195.200750548023) / 17624.835054167113 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in Burgers

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test1 - Duration of single invocation	3.77 secs	280.38 ms	0.07	0.26	True
Test2 - Duration of single invocation	846.72 ms	279.59 ms	0.33	0.00	True
Test0 - Duration of single invocation	4.34 secs	327.65 ms	0.08	0.25	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Burgers*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Burgers* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'Burgers*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter Burgers* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

Burgers.Test1

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 280.3770264 < 3.670487001260714.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 461.2292535136842 (T) = (0 -280388509.074026) / Math.Sqrt((18055981294840170 / (299)) + (953539034.5325089 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.927446716931794 = (3864587475.8064137 - 280388509.074026) / 3864587475.8064137 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Burgers.Test2

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 279.5924904666666 < 802.2220948483333.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 1870.6380535211442 (T) = (0 -279589935.66683316) / Math.Sqrt((27506801465915.445 / (299)) + (230375676.0839924 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6699191745093585 = (847034768.6850418 - 279589935.66683316) / 847034768.6850418 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Burgers.Test0

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 327.6497176923077 < 3.9415248136.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 440.32380987567154 (T) = (0 -328090943.68831176) / Math.Sqrt((23661079884098110 / (299)) + (267262677727.36255 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9227238508831185 = (4245694789.8900642 - 328090943.68831176) / 4245694789.8900642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Plane

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DotCoordinateBenchmark - Duration of single invocation	18.21 ns	0.05 ns	0.00	0.06	True
CreateFromVerticesBenchmark - Duration of single invocation	128.39 ns	5.99 ns	0.05	0.01	False
DotNormalBenchmark - Duration of single invocation	17.72 ns	0.07 ns	0.00	0.27	True
CreateFromVector3WithScalarDBenchmark - Duration of single invocation	18.36 ns	0.00 ns	0.00	0.15	True
InequalityOperatorBenchmark - Duration of single invocation	7.54 ns	0.06 ns	0.01	0.25	False
DotBenchmark - Duration of single invocation	22.69 ns	0.00 ns	0.00	0.06	True
EqualityOperatorBenchmark - Duration of single invocation	7.08 ns	0.03 ns	0.00	0.21	False
TransformByMatrix4x4Benchmark - Duration of single invocation	223.77 ns	41.92 ns	0.19	0.02	True
CreateFromScalarXYZDBenchmark - Duration of single invocation	16.12 ns	0.06 ns	0.00	0.29	True
TransformByQuaternionBenchmark - Duration of single invocation	58.25 ns	0.02 ns	0.00	0.08	True
CreateFromVector4Benchmark - Duration of single invocation	35.45 ns	0.02 ns	0.00	0.10	True
NormalizeBenchmark - Duration of single invocation	49.36 ns	0.03 ns	0.00	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Plane* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.051753821573786454 < 17.23307029346255.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 185.1818493094696 (T) = (0 -0.05441865152559206) / Math.Sqrt((2.604602049114624 / (299)) + (0.007167671804678105 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9969721595150055 = (17.972760386579687 - 0.05441865152559206) / 17.972760386579687 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.992118667689723 < 121.91603306020994.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 64.51365440827323 (T) = (0 -6.187696104791282) / Math.Sqrt((517.8738047764559 / (299)) + (0.009800798457623324 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.932088047315239 = (91.11350594664559 - 6.187696104791282) / 91.11350594664559 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.DotNormalBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.07096054193127564 < 16.660465654815646.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 186.57433943142328 (T) = (0 -0.09324963436086219) / Math.Sqrt((2.3233447993749925 / (299)) + (0.006915913668857988 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9945760365679894 = (17.192157640763394 - 0.09324963436086219) / 17.192157640763394 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromVector3WithScalarDBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.49539041626722.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 179.56503014660302 (T) = (0 -0.13018804326394118) / Math.Sqrt((2.5143320667146676 / (299)) + (0.015497082437551482 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9927360568468236 = (17.922503042581408 - 0.13018804326394118) / 17.922503042581408 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06415204018345916 < 7.218078737492155.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 73.28044171440513 (T) = (0 -0.06639196451715558) / Math.Sqrt((4.109242847845592 / (299)) + (0.01653017833759695 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9927159312035592 = (9.114681144911208 - 0.06639196451715558) / 9.114681144911208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.DotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 21.644771921502105.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 292.09955005945085 (T) = (0 -0.06386076221720363) / Math.Sqrt((1.6091766491875898 / (299)) + (0.0069888493340142955 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.997189502106847 = (22.722223835421328 - 0.06386076221720363) / 22.722223835421328 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.03054812879961957 < 6.883046100663516.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 113.08991622189285 (T) = (0 -0.046146790072666304) / Math.Sqrt((1.3743468282116313 / (299)) + (0.007484011260325982 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9944140402389121 = (8.261210614893415 - 0.046146790072666304) / 8.261210614893415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.TransformByMatrix4x4Benchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.91927788159267 < 212.60106862578638.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 463.13770916988045 (T) = (0 -42.536556148418974) / Math.Sqrt((48.175044230143456 / (299)) + (0.07481938589134637 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.8169073935907181 = (232.3226316049788 - 42.536556148418974) / 232.3226316049788 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromScalarXYZDBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06487007884952399 < 15.273667254571917.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 264.43286104184523 (T) = (0 -0.06897486954949449) / Math.Sqrt((1.0673026085877744 / (299)) + (0.004429274099740894 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9958784145893679 = (16.73503341009605 - 0.06897486954949449) / 16.73503341009605 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.022749267564745736 < 55.807598405750845.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 262.6453191280113 (T) = (0 -0.06683066561650505) / Math.Sqrt((17.604624110442717 / (299)) + (0.007679980379553923 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9989586036727163 = (64.17409382537795 - 0.06683066561650505) / 64.17409382537795 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.CreateFromVector4Benchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01598567265046228 < 33.857725596972465.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 301.01670151595897 (T) = (0 -0.10630532375754133) / Math.Sqrt((4.125384636469135 / (299)) + (0.013091481876352281 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9971235942166015 = (36.95769365055953 - 0.10630532375754133) / 36.95769365055953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Plane.NormalizeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.03420303266069849 < 46.227938922800504.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 86.57091352334157 (T) = (0 -0.37161981904919733) / Math.Sqrt((84.20229899850423 / (299)) + (0.13755454060226685 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.992146883524285 = (47.32131761936818 - 0.37161981904919733) / 47.32131761936818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Constructor

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConstructorBenchmark_UInt16 - Duration of single invocation	19.40 ns	0.67 ns	0.03	0.06	True
ConstructorBenchmark_Single - Duration of single invocation	20.62 ns	0.67 ns	0.03	0.05	True
ConstructorBenchmark_Double - Duration of single invocation	19.50 ns	0.72 ns	0.04	0.06	True
SpanCastBenchmark_Byte - Duration of single invocation	9.36 ns	0.50 ns	0.05	0.11	True
SpanCastBenchmark_UInt64 - Duration of single invocation	9.60 ns	0.78 ns	0.08	0.07	True
SpanCastBenchmark_Int32 - Duration of single invocation	9.61 ns	0.71 ns	0.07	0.11	True
SpanCastBenchmark_SByte - Duration of single invocation	9.15 ns	0.60 ns	0.07	0.13	True
ConstructorBenchmark_Int32 - Duration of single invocation	19.68 ns	0.61 ns	0.03	0.09	True
ConstructorBenchmark_UInt64 - Duration of single invocation	19.73 ns	0.60 ns	0.03	0.06	True
SpanCastBenchmark_Double - Duration of single invocation	10.16 ns	0.93 ns	0.09	0.11	True
ConstructorBenchmark_Byte - Duration of single invocation	19.01 ns	0.61 ns	0.03	0.06	True
SpanCastBenchmark_UInt16 - Duration of single invocation	9.46 ns	0.75 ns	0.08	0.10	True
ConstructorBenchmark_UInt32 - Duration of single invocation	19.77 ns	0.82 ns	0.04	0.06	True
ConstructorBenchmark_Int64 - Duration of single invocation	19.90 ns	0.58 ns	0.03	0.05	True
ConstructorBenchmark_SByte - Duration of single invocation	19.55 ns	0.81 ns	0.04	0.07	True
SpanCastBenchmark_Single - Duration of single invocation	9.71 ns	0.69 ns	0.07	0.09	True
SpanCastBenchmark_Int64 - Duration of single invocation	9.65 ns	0.74 ns	0.08	0.12	True
SpanCastBenchmark_Int16 - Duration of single invocation	13.20 ns	0.72 ns	0.05	0.09	True
ConstructorBenchmark_Int16 - Duration of single invocation	19.70 ns	0.60 ns	0.03	0.07	True
SpanCastBenchmark_UInt32 - Duration of single invocation	9.56 ns	0.75 ns	0.08	0.12	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Constructor*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Constructor* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Constructor*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Constructor* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt16

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.667144073857649 < 18.847896773764866.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 279.1598347005488 (T) = (0 -0.6291402459383112) / Math.Sqrt((1.4913445999320745 / (299)) + (0.0009841578542072698 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9693410590515082 = (20.52061246979449 - 0.6291402459383112) / 20.52061246979449 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Single

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6672207539132913 < 19.026166397761397.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 276.2042608742738 (T) = (0 -0.6203959836390373) / Math.Sqrt((1.5092897534516105 / (299)) + (0.001565554634994134 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9697645511969932 = (20.518828335610564 - 0.6203959836390373) / 20.518828335610564 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Double

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.72425641892392 < 18.673340574539566.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 271.72348206526027 (T) = (0 -0.7992885474198844) / Math.Sqrt((1.4501238775788667 / (299)) + (0.0044598722706974195 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9610056826155259 = (20.497564799996375 - 0.7992885474198844) / 20.497564799996375 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Byte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.4965048264414351 < 9.080327589015958.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 199.07653164973962 (T) = (0 -0.4307303084591426) / Math.Sqrt((0.4436536193362972 / (299)) + (0.008070262298588916 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9560712783328521 = (9.805209259737339 - 0.4307303084591426) / 9.805209259737339 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt64

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7833650783455101 < 9.175311984228003.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 237.31593423420102 (T) = (0 -0.7317898512291715) / Math.Sqrt((0.44500411834155634 / (299)) + (0.0013324204309538012 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9286224492789233 = (10.252381089522116 - 0.7317898512291715) / 10.252381089522116 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Int32

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.711922832611063 < 9.195758332843683.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 200.73531647184276 (T) = (0 -0.709932821853432) / Math.Sqrt((0.6112333473575255 / (299)) + (0.002772260530894244 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9312685127683873 = (10.329076969644008 - 0.709932821853432) / 10.329076969644008 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_SByte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6010864167623596 < 8.68329487832613.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 197.13107944845459 (T) = (0 -0.6513964743689732) / Math.Sqrt((0.5241391277106354 / (299)) + (0.0042051414652826465 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.933262660850905 = (9.760600028025042 - 0.6513964743689732) / 9.760600028025042 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Int32

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6134300865050084 < 18.65489152504511.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 275.70628623709155 (T) = (0 -0.648748324927914) / Math.Sqrt((1.4639235372792287 / (299)) + (0.0033437683202165087 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9684004397951685 = (20.530296014332592 - 0.648748324927914) / 20.530296014332592 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt64

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.5984107965294422 < 18.900493100556375.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 283.5047374380677 (T) = (0 -0.6411293845064775) / Math.Sqrt((1.383096976789597 / (299)) + (0.0029587066758441525 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9686883676516476 = (20.47575729600095 - 0.6411293845064775) / 20.47575729600095 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Double

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.9276427016535587 < 9.179870490036413.
IsChangePoint: Marked as a change because one of 3/20/2023 11:25:43 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 185.87196922353178 (T) = (0 -0.9597510235627926) / Math.Sqrt((0.5882863976368777 / (299)) + (0.006101340695570108 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9067703752660864 = (10.294485538282657 - 0.9597510235627926) / 10.294485538282657 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Byte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6133675782323867 < 18.559306490672654.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 277.5951128831732 (T) = (0 -0.5991918834812453) / Math.Sqrt((1.4978136344438224 / (299)) + (0.0011576141329886695 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.970702380515962 = (20.45189657158663 - 0.5991918834812453) / 20.45189657158663 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt16

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7476771174374175 < 9.181827734815924.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 203.00253920784021 (T) = (0 -0.705860838791372) / Math.Sqrt((0.6344280449574663 / (299)) + (0.001231251198370362 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9314719061560541 = (10.300313334247672 - 0.705860838791372) / 10.300313334247672 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt32

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8182707988995962 < 18.744242298979035.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 270.38612909776447 (T) = (0 -0.6346340239988394) / Math.Sqrt((1.4927969749257204 / (299)) + (0.004661334708793712 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9690935106868188 = (20.53400557947476 - 0.6346340239988394) / 20.53400557947476 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Int64

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.5788004060912545 < 18.733648215503585.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 9:01:12 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 280.36685621261466 (T) = (0 -0.6217450058864803) / Math.Sqrt((1.386216761366909 / (299)) + (0.004083027155759139 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.969613556494808 = (20.46126279241086 - 0.6217450058864803) / 20.46126279241086 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_SByte

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8101299121838719 < 18.717618479982242.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 278.73092087359714 (T) = (0 -0.7923335282541508) / Math.Sqrt((1.4357694555149503 / (299)) + (0.002388752645378402 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9614232066906836 = (20.53912366175858 - 0.7923335282541508) / 20.53912366175858 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Single

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6936862770095633 < 9.279340489064518.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 199.73247390934304 (T) = (0 -0.7225824878824778) / Math.Sqrt((0.6514097890727418 / (299)) + (0.0013480058611976496 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9298726404534091 = (10.303859899393652 - 0.7225824878824778) / 10.303859899393652 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Int64

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7409481851475945 < 9.317808263449969.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 187.20005958374767 (T) = (0 -0.7782508073472154) / Math.Sqrt((0.7481559551063348 / (299)) + (0.0014745880819292802 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9250956303601744 = (10.38992532864771 - 0.7782508073472154) / 10.38992532864771 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_Int16

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7224555199869779 < 9.383514142728542.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 217.6691797730618 (T) = (0 -0.737968958690324) / Math.Sqrt((0.49624780958848685 / (299)) + (0.0028784596382723003 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.928207244105388 = (10.279156295011488 - 0.737968958690324) / 10.279156295011488 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.ConstructorBenchmark_Int16

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.5996339286765188 < 18.83809268892845.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 276.97874855176326 (T) = (0 -0.6315809908405383) / Math.Sqrt((1.5174516326014222 / (299)) + (0.0009984506937937903 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9692498567194971 = (20.539123511694118 - 0.6315809908405383) / 20.539123511694118 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt32

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7520729081327231 < 9.094373933668745.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 191.88810961265028 (T) = (0 -0.6874949525925556) / Math.Sqrt((0.654184001761377 / (299)) + (0.003633778924228783 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9333619715968082 = (10.316856141554538 - 0.6874949525925556) / 10.316856141554538 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ForEach - Duration of single invocation	517.66 ns	243.89 ns	0.47	0.04	True
GetCredential_HostPort - Duration of single invocation	238.78 ns	78.55 ns	0.33	0.10	True
GetCredential_Uri - Duration of single invocation	423.01 ns	147.16 ns	0.35	0.01	False
GetCredential_Uri - Duration of single invocation	2.48 μs	954.15 ns	0.38	0.01	True
GetCredential_HostPort - Duration of single invocation	71.07 ns	20.80 ns	0.29	0.45	True
ForEach - Duration of single invocation	425.63 ns	214.25 ns	0.50	0.02	True
ForEach - Duration of single invocation	108.58 ns	41.00 ns	0.38	0.44	True
ForEach - Duration of single invocation	867.34 ns	400.15 ns	0.46	0.02	True
GetCredential_HostPort - Duration of single invocation	194.69 ns	61.72 ns	0.32	0.14	True
GetCredential_Uri - Duration of single invocation	1.91 μs	714.12 ns	0.37	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Net.Primitives.Tests.CredentialCacheTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Net.Primitives.Tests.CredentialCacheTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 243.89323164476713 < 498.24781757546515.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 60.635394376860496 (T) = (0 -250.25913706542576) / Math.Sqrt((249.23452524086548 / (299)) + (244.12562533039394 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5376066476353605 = (541.2256378376164 - 250.25913706542576) / 541.2256378376164 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 78.54559730403864 < 230.5032835106227.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 130.03358349445787 (T) = (0 -77.158762181342) / Math.Sqrt((25.68145980805085 / (299)) + (16.678346358574135 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6808295373906659 = (241.74781573000573 - 77.158762181342) / 241.74781573000573 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 147.16168100384945 < 399.11481760661866.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 109.55481437698508 (T) = (0 -145.70439264222156) / Math.Sqrt((2054.066516145184 / (299)) + (3.663041914116369 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6686486581930109 = (439.72778817685867 - 145.70439264222156) / 439.72778817685867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://name5", uriCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 954.1460507885115 < 2.355685984804337.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 331.11552520791133 (T) = (0 -917.0898145815362) / Math.Sqrt((1122.7921004432785 / (299)) + (201.635181127649 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6291858000230972 = (2473.1787904526304 - 917.0898145815362) / 2473.1787904526304 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.80414229227379 < 67.94901770469781.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 58.815568699735756 (T) = (0 -22.873013041711545) / Math.Sqrt((13.295381346634128 / (299)) + (7.877917762868262 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.691608855124001 = (74.16883857319755 - 22.873013041711545) / 74.16883857319755 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 214.2471885333616 < 396.72174636710554.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 31.077256295473894 (T) = (0 -235.2867645704908) / Math.Sqrt((18.556505863704857 / (299)) + (399.0079603642036 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.44326267167775496 = (422.6171887549536 - 235.2867645704908) / 422.6171887549536 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.99809692010982 < 104.3249549387765.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 48.21318442434019 (T) = (0 -41.06707016740026) / Math.Sqrt((20.13020255035958 / (299)) + (23.832903400277036 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6369873673463227 = (113.12848775314997 - 41.06707016740026) / 113.12848775314997 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 400.1501972469406 < 830.0750859982746.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 78.22054950517106 (T) = (0 -406.5359859222723) / Math.Sqrt((302.4973282997098 / (299)) + (407.65002150373607 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5427909673370295 = (889.1687540695382 - 406.5359859222723) / 889.1687540695382 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.72323903655201 < 186.75342409519178.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 208.80450927442683 (T) = (0 -58.64341435658978) / Math.Sqrt((20.26376822131138 / (299)) + (4.202074038326117 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.7048337463153019 = (198.67926507355318 - 58.64341435658978) / 198.67926507355318 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 714.1230871561396 < 1.839577086719849.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 301.21545383545606 (T) = (0 -713.2674206504536) / Math.Sqrt((2794.044264977744 / (299)) + (85.51064670471739 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6359998727139524 = (1959.5251956874622 - 713.2674206504536) / 1959.5251956874622 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Tests.Perf_Guid

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ctor_str - Duration of single invocation	589.20 ns	29.75 ns	0.05	0.03	True
TryWriteBytes - Duration of single invocation	11.89 ns	0.00 ns	0.00	0.12	True
GuidToString - Duration of single invocation	232.40 ns	24.83 ns	0.11	0.03	False
EqualsSame - Duration of single invocation	12.20 ns	0.05 ns	0.00	0.34	False
ctor_bytes - Duration of single invocation	23.31 ns	1.84 ns	0.08	0.02	True
NewGuid - Duration of single invocation	552.12 ns	460.71 ns	0.83	0.20	False
EqualsNotSame - Duration of single invocation	11.69 ns	0.49 ns	0.04	0.53	False
Parse - Duration of single invocation	613.72 ns	30.46 ns	0.05	0.02	True
ParseExactD - Duration of single invocation	603.46 ns	33.27 ns	0.06	0.03	True
EqualsOperator - Duration of single invocation	13.19 ns	0.20 ns	0.02	0.25	False
NotEqualsOperator - Duration of single invocation	14.84 ns	0.55 ns	0.04	0.25	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Guid*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Guid* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Tests.Perf_Guid*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Tests.Perf_Guid* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Guid.ctor_str

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.754255817098432 < 556.8265739837033.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 585.8778630192276 (T) = (0 -30.25062242749472) / Math.Sqrt((283.67536518680924 / (299)) + (0.14372705866059787 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9499851806602033 = (604.8331839804197 - 30.25062242749472) / 604.8331839804197 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.TryWriteBytes

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 10.886240685045847.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 138.66188835586178 (T) = (0 -0.14801883625242612) / Math.Sqrt((1.7609406277174915 / (299)) + (0.02434732689133159 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9882801602966079 = (12.62976627654599 - 0.14801883625242612) / 12.62976627654599 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.GuidToString

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.82734759665347 < 221.71099256920033.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 45.97743811031215 (T) = (0 -25.405706758972883) / Math.Sqrt((1841.5974808627764 / (299)) + (1.07863055173939 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.8190681824873551 = (140.4159152781259 - 25.405706758972883) / 140.4159152781259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.EqualsSame

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.05479908209585817 < 11.604438562642095.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 31.713325596838224 (T) = (0 -0.03861117026396585) / Math.Sqrt((12.728521079486558 / (299)) + (0.008461578231185242 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9941857118387035 = (6.640739019607884 - 0.03861117026396585) / 6.640739019607884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.ctor_bytes

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8407518390822752 < 21.630662778840595.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 109.12683894510644 (T) = (0 -2.645402183853633) / Math.Sqrt((0.17739295419899032 / (299)) + (0.3701669896960976 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.8841753455127063 = (22.83971573723833 - 2.645402183853633) / 22.83971573723833 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.NewGuid

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 460.7125745986161 < 528.2514233768467.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 32.67553394318054 (T) = (0 -464.30560813397403) / Math.Sqrt((396.8531580875593 / (299)) + (100.11515310768362 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.18518011104427237 = (569.8260614735701 - 464.30560813397403) / 569.8260614735701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.EqualsNotSame

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.4921474460310427 < 11.681822077936513.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 19.62015607289607 (T) = (0 -0.11914839230836359) / Math.Sqrt((18.94759892225374 / (299)) + (0.030935540337559787 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9769387726910572 = (5.1666110702685755 - 0.11914839230836359) / 5.1666110702685755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.Parse

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.45707412490482 < 584.144496766261.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 601.0902867838062 (T) = (0 -30.395110696032717) / Math.Sqrt((285.85777420585873 / (299)) + (0.5042494733590578 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9519104452243393 = (632.052237493711 - 30.395110696032717) / 632.052237493711 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.ParseExactD

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.265211436974916 < 564.2199135648375.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 1014.6672835144965 (T) = (0 -34.0442200961251) / Math.Sqrt((86.22834464858445 / (299)) + (0.290522412762657 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9435741342338092 = (603.3442222613382 - 34.0442200961251) / 603.3442222613382 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.EqualsOperator

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.20024782081343556 < 12.680026390406297.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 40.86315241686063 (T) = (0 -0.18512320362650406) / Math.Sqrt((11.885686938341612 / (299)) + (0.0322120712530697 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9785416194085347 = (8.627081751925617 - 0.18512320362650406) / 8.627081751925617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Guid.NotEqualsOperator

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.5473037303866559 < 13.01099479612656.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 39.77768716544067 (T) = (0 -0.26811645302732) / Math.Sqrt((12.65333045019877 / (299)) + (0.025584782509463992 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9690855860097554 = (8.67286221604998 - 0.26811645302732) / 8.67286221604998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SquareRootBenchmark - Duration of single invocation	25.83 ns	0.41 ns	0.02	0.35	True
GetHashCodeBenchmark - Duration of single invocation	29.89 ns	12.83 ns	0.43	0.03	True
ZeroBenchmark - Duration of single invocation	1.23 ns	0.00 ns	0.00	0.61	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Double>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_VectorOf<Double>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.41134446024352944 < 22.542451454857186.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 340.53884125724085 (T) = (0 -0.445563601914523) / Math.Sqrt((1.3973253795812923 / (299)) + (0.0009620791964034316 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9813900521633345 = (23.9422273412647 - 0.445563601914523) / 23.9422273412647 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.829245993990824 < 28.4421356386339.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 386.2098953217284 (T) = (0 -12.7976356100938) / Math.Sqrt((0.5705579143602088 / (299)) + (0.002480196082929473 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5822872810784707 = (30.637409469204933 - 12.7976356100938) / 30.637409469204933 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_VectorOf<Double>.ZeroBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 1.1123379528797737.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 51.246788630290666 (T) = (0 -0.011877004960697806) / Math.Sqrt((0.05867805498404746 / (299)) + (0.0015516957152008433 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9875382508011997 = (0.9530768731761367 - 0.011877004960697806) / 0.9530768731761367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	33a30bc0b011960e66fdf86d5833bfdd7f60102f
Compare	c6e435b135dab4254baee6e3d3d00672626a8838
Diff	Diff
Configs	CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono

Improvements in System.Numerics.Tests.Perf_Vector3

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ClampBenchmark - Duration of single invocation	52.82 ns	0.02 ns	0.00	0.05	True
MaxBenchmark - Duration of single invocation	24.48 ns	0.23 ns	0.01	0.06	True
CrossBenchmark - Duration of single invocation	15.63 ns	0.00 ns	0.00	0.10	True
UnitYBenchmark - Duration of single invocation	8.48 ns	0.00 ns	0.00	0.18	False
DivideByVector3OperatorBenchmark - Duration of single invocation	15.63 ns	0.24 ns	0.02	0.11	False
DivideByScalarOperatorBenchmark - Duration of single invocation	18.92 ns	0.00 ns	0.00	0.06	False
ReflectBenchmark - Duration of single invocation	55.38 ns	0.85 ns	0.02	0.03	False
SubtractOperatorBenchmark - Duration of single invocation	11.77 ns	0.07 ns	0.01	0.15	False
DivideByVector3Benchmark - Duration of single invocation	14.97 ns	0.24 ns	0.02	0.10	False
AbsBenchmark - Duration of single invocation	48.01 ns	0.00 ns	0.00	0.03	True
LerpBenchmark - Duration of single invocation	46.60 ns	0.12 ns	0.00	0.03	False
NormalizeBenchmark - Duration of single invocation	51.33 ns	0.76 ns	0.01	0.19	False
LengthSquaredBenchmark - Duration of single invocation	6.80 ns	0.23 ns	0.03	0.22	True
EqualityOperatorBenchmark - Duration of single invocation	1.49 ns	0.00 ns	0.00	0.65	False
LengthBenchmark - Duration of single invocation	17.98 ns	0.05 ns	0.00	0.31	True
MinBenchmark - Duration of single invocation	24.27 ns	0.23 ns	0.01	0.10	True
MultiplyOperatorBenchmark - Duration of single invocation	12.37 ns	0.00 ns	0.00	0.18	False
DivideByScalarBenchmark - Duration of single invocation	21.50 ns	0.00 ns	0.00	0.06	False
SubtractFunctionBenchmark - Duration of single invocation	11.66 ns	0.06 ns	0.01	0.18	False
NegateBenchmark - Duration of single invocation	10.00 ns	0.02 ns	0.00	0.10	False
TransformByMatrix4x4Benchmark - Duration of single invocation	213.00 ns	0.39 ns	0.00	0.14	False
MultiplyByScalarOperatorBenchmark - Duration of single invocation	17.19 ns	0.00 ns	0.00	0.09	False
NegateOperatorBenchmark - Duration of single invocation	10.46 ns	0.10 ns	0.01	0.10	False
SquareRootBenchmark - Duration of single invocation	46.13 ns	0.00 ns	0.00	0.65	False
MultiplyFunctionBenchmark - Duration of single invocation	12.43 ns	0.17 ns	0.01	0.13	False
TransformNormalByMatrix4x4Benchmark - Duration of single invocation	201.03 ns	0.08 ns	0.00	0.04	False
DistanceSquaredBenchmark - Duration of single invocation	15.55 ns	2.26 ns	0.15	0.05	False
MultiplyByScalarBenchmark - Duration of single invocation	20.79 ns	0.24 ns	0.01	0.06	False
DistanceBenchmark - Duration of single invocation	27.05 ns	0.07 ns	0.00	0.06	True
CreateFromScalar - Duration of single invocation	9.11 ns	0.00 ns	0.00	0.24	True
UnitZBenchmark - Duration of single invocation	8.83 ns	0.00 ns	0.00	0.13	False
CreateFromScalarXYZBenchmark - Duration of single invocation	6.54 ns	0.00 ns	0.00	0.44	True
ZeroBenchmark - Duration of single invocation	2.38 ns	0.23 ns	0.10	0.58	False
CreateFromVector2WithScalarBenchmark - Duration of single invocation	10.86 ns	0.00 ns	0.00	0.17	True
UnitXBenchmark - Duration of single invocation	9.05 ns	0.10 ns	0.01	0.16	False
DotBenchmark - Duration of single invocation	5.89 ns	0.05 ns	0.01	0.38	False
AddOperatorBenchmark - Duration of single invocation	11.80 ns	0.03 ns	0.00	0.21	False
GetHashCodeBenchmark - Duration of single invocation	12.58 ns	4.23 ns	0.34	0.09	False
InequalityOperatorBenchmark - Duration of single invocation	3.67 ns	0.00 ns	0.00	0.51	False
EqualsBenchmark - Duration of single invocation	26.94 ns	0.46 ns	0.02	0.15	False
TransformByQuaternionBenchmark - Duration of single invocation	40.64 ns	0.00 ns	0.00	0.05	True
AddFunctionBenchmark - Duration of single invocation	12.20 ns	0.02 ns	0.00	0.17	False
OneBenchmark - Duration of single invocation	11.93 ns	0.06 ns	0.01	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

Repro Steps

Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))

• Libraries build extracted to runtime/artifacts or build instructions: Libraries README args: -subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0
• CoreCLR product build extracted to runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args: -subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0
• AOT MONO build extracted to runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args: -arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false
• Dotnet SDK installed for dotnet commands
• Running commands from the runtime folder

Linux

# Set $RunDir to the runtime directory
RunDir=`pwd`

# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'

# Create aot directory 
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack

# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance

# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector3* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200

Windows

# Set $RunDir to the runtime directory
$RunDir="FullPathHere"

# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'

# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y

# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release

# Clone performance 
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance

# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime  --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"

# Individual Commands:
# Restore 
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Numerics.Tests.Perf_Vector3* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.ClampBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.017677385505233307 < 49.81880616868741.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 227.72841743997498 (T) = (0 -0.06158433331950959) / Math.Sqrt((19.987879347359687 / (299)) + (0.004978444104443398 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9989586719120627 = (59.14018265030604 - 0.06158433331950959) / 59.14018265030604 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.2326018526898823 < 23.334395220672267.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 214.66383277385296 (T) = (0 -0.08719534304990975) / Math.Sqrt((4.796636443317583 / (299)) + (0.007140738860923676 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9968658161616135 = (27.820749370847153 - 0.08719534304990975) / 27.820749370847153 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CrossBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 14.194613714137388.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 154.8286868093284 (T) = (0 -0.06867856526977995) / Math.Sqrt((3.541712880107432 / (299)) + (0.01157255696353803 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9961092948971932 = (17.651958566645813 - 0.06867856526977995) / 17.651958566645813 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitYBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.36558175793009.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 116.32574869165506 (T) = (0 -0.08640740561662713) / Math.Sqrt((1.7530282526905232 / (299)) + (0.012345298437477666 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9911907138684646 = (9.808672839823783 - 0.08640740561662713) / 9.808672839823783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.23836732912173436 < 14.265640511818653.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 147.56469898407912 (T) = (0 -0.07542836605385377) / Math.Sqrt((4.1765046251659905 / (299)) + (0.010276906670113658 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9958302705397474 = (18.089510787897098 - 0.07542836605385377) / 18.089510787897098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 17.90687084617409.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 77.35318594238794 (T) = (0 -0.07167399903286108) / Math.Sqrt((40.42416854836337 / (299)) + (0.01168613222719993 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9974961400016199 = (28.625402010986924 - 0.07167399903286108) / 28.625402010986924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ReflectBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8546964380832308 < 52.527232396394574.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 79.31773980927898 (T) = (0 -0.7786879678169618) / Math.Sqrt((327.7651342323566 / (299)) + (0.004764234249522632 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9907122798999255 = (83.84059375462002 - 0.7786879678169618) / 83.84059375462002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0676956616722549 < 11.187034242616201.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 104.68613129520057 (T) = (0 -0.12814241363324463) / Math.Sqrt((5.959482163094262 / (299)) + (0.012291520114545101 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9916335465644448 = (15.31621667655194 - 0.12814241363324463) / 15.31621667655194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.24017467145648091 < 14.565854366699586.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 149.56084407965795 (T) = (0 -0.12560495682619294) / Math.Sqrt((4.0606117700118505 / (299)) + (0.008884036960738114 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9930472863508047 = (18.065601887794134 - 0.12560495682619294) / 18.065601887794134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.003161132209016316 < 45.391249468933715.
IsChangePoint: Marked as a change because one of 3/8/2023 4:26:04 PM, 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 197.32849563264014 (T) = (0 -0.08058683945882691) / Math.Sqrt((19.011003878290705 / (299)) + (0.010327518817679477 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.998394809059217 = (50.20389625393197 - 0.08058683945882691) / 50.20389625393197 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.11574483932896748 < 44.285727005403054.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 98.65511134653161 (T) = (0 -0.12542496380493365) / Math.Sqrt((127.13636253068464 / (299)) + (0.00953436181877089 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9980560824139383 = (64.52174963807967 - 0.12542496380493365) / 64.52174963807967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.7593957848124903 < 48.654323733723636.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 132.0232629557865 (T) = (0 -0.9981446967963831) / Math.Sqrt((63.466879603415414 / (299)) + (0.1313909881494623 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9842841616380656 = (63.51202359105481 - 0.9981446967963831) / 63.51202359105481 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.2277463664857789 < 6.462310172016793.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 169.41519098361698 (T) = (0 -0.0959648459957328) / Math.Sqrt((0.4292000510182191 / (299)) + (0.004357868902669526 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9869373428770926 = (7.346502713252948 - 0.0959648459957328) / 7.346502713252948 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 1.4779645239400119.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 29.24813596902422 (T) = (0 -0.08623163491648879) / Math.Sqrt((0.39066441251826795 / (299)) + (0.03662294199065925 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.958495961327621 = (2.0776685275661197 - 0.08623163491648879) / 2.0776685275661197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.LengthBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.0540246977905653 < 16.611979133704622.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 361.2634969867363 (T) = (0 -0.1324472207814611) / Math.Sqrt((0.40513015190311447 / (299)) + (0.011064576090545692 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9925110214589687 = (17.685618947336742 - 0.1324472207814611) / 17.685618947336742 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.23215709818801808 < 22.96770502175717.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 284.1794144362287 (T) = (0 -0.15993083611854084) / Math.Sqrt((2.425852606994094 / (299)) + (0.004734944302724221 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9939482474730202 = (26.427193677457954 - 0.15993083611854084) / 26.427193677457954 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 11.804997816511177.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 114.14321749453858 (T) = (0 -0.04436122841799703) / Math.Sqrt((5.547337287009903 / (299)) + (0.0069284613515426755 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9972017907309367 = (15.853434876530336 - 0.04436122841799703) / 15.853434876530336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 20.45379101199504.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 89.115359522531 (T) = (0 -0.06737134584323373) / Math.Sqrt((33.74232091656348 / (299)) + (0.005034975554278362 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.997759125613122 = (30.064757863154664 - 0.06737134584323373) / 30.064757863154664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06287189179977083 < 11.390817616651711.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 108.62381528990004 (T) = (0 -0.07929592577459989) / Math.Sqrt((5.674458730913003 / (299)) + (0.007291300227224039 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9948181115847553 = (15.302515110383084 - 0.07929592577459989) / 15.302515110383084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.01698846941287281 < 9.557428801074668.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 78.33859315182924 (T) = (0 -0.04684215910624181) / Math.Sqrt((11.270815721209791 / (299)) + (0.0028482945705356416 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9969401396618809 = (15.308593834395815 - 0.04684215910624181) / 15.308593834395815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.39313889177977557 < 202.38218930962395.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 112.16534591943098 (T) = (0 -0.42429746173051497) / Math.Sqrt((1857.0115824129066 / (299)) + (0.013180638928565743 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9984845574264481 = (279.9825405037225 - 0.42429746173051497) / 279.9825405037225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 16.351690529327946.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 73.15500977075878 (T) = (0 -0.061305643456194596) / Math.Sqrt((40.942509033635424 / (299)) + (0.008935162491351498 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9977471052728509 = (27.211943246798842 - 0.061305643456194596) / 27.211943246798842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.09548184353805068 < 9.762004196838605.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 78.86625667945671 (T) = (0 -0.01774713401132749) / Math.Sqrt((11.186924851633284 / (299)) + (0.0015599609504406882 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9988401769773184 = (15.301587970115508 - 0.01774713401132749) / 15.301587970115508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 49.79859685910814.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 138.64397683454973 (T) = (0 -0.08407053287415735) / Math.Sqrt((46.097624838315966 / (299)) + (0.010735602121715895 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9984629048012371 = (54.69442162191304 - 0.08407053287415735) / 54.69442162191304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.174014073390025 < 11.791626019595272.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 111.71791498386028 (T) = (0 -0.05967156945513186) / Math.Sqrt((5.648017243644698 / (299)) + (0.010745730172300936 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9962247954101539 = (15.806181634665784 - 0.05967156945513186) / 15.806181634665784 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.07732326806874917 < 191.01737993349133.
IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 125.62796529970146 (T) = (0 -0.09706520883474844) / Math.Sqrt((1249.9680192942053 / (299)) + (0.011537352919711887 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.999622301835792 = (256.9914763507203 - 0.09706520883474844) / 256.9914763507203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.263626627724315 < 14.549951731076916.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 74.04465543562465 (T) = (0 -0.2933565470883981) / Math.Sqrt((8.641125583619681 / (299)) + (0.4336116318199911 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9850687276266403 = (19.647123148848568 - 0.2933565470883981) / 19.647123148848568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.23595719296020226 < 19.42291353116479.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 87.18843877757091 (T) = (0 -0.042786168981169874) / Math.Sqrt((33.17685284150847 / (299)) + (0.0063129498075154295 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9985327484418645 = (29.160758933211238 - 0.042786168981169874) / 29.160758933211238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DistanceBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06729692382383703 < 25.43438788586415.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 173.87442637979134 (T) = (0 -0.07624137883324048) / Math.Sqrt((9.493035024907986 / (299)) + (0.007588059758076476 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9975713520170973 = (31.392519364671617 - 0.07624137883324048) / 31.392519364671617 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalar

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.648732382049797.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 145.6251856215186 (T) = (0 -0.06051003743510384) / Math.Sqrt((1.1527629210095338 / (299)) + (0.009118166434038923 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.993965531275006 = (10.027400951550067 - 0.06051003743510384) / 10.027400951550067 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitZBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 8.201062947685076.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 119.51711928151822 (T) = (0 -0.07658213117802842) / Math.Sqrt((1.805520267232739 / (299)) + (0.00729060181231804 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9922333940013485 = (9.86043726066767 - 0.07658213117802842) / 9.86043726066767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 6.129297814788982.
IsChangePoint: Marked as a change because one of 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 207.46790724690362 (T) = (0 -0.011307418087731879) / Math.Sqrt((0.24132612371741022 / (299)) + (0.0013837628653712753 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9982187741570102 = (6.348110281598103 - 0.011307418087731879) / 6.348110281598103 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ZeroBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.23253231296612303 < 2.2846002765462647.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 56.57018668382046 (T) = (0 -0.04324786648447723) / Math.Sqrt((0.1712857649765539 / (299)) + (0.008763238782121744 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9797599060617637 = (2.136742379578379 - 0.04324786648447723) / 2.136742379578379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 10.192847154247765.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 188.91745273206055 (T) = (0 -0.09759833919962514) / Math.Sqrt((0.6604024286018544 / (299)) + (0.015351702797068337 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9914682542826027 = (11.439433667204687 - 0.09759833919962514) / 11.439433667204687 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitXBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.09611400820420042 < 8.30858257725792.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 120.50555783052643 (T) = (0 -0.11263926531189204) / Math.Sqrt((1.73257038819344 / (299)) + (0.00795626290081593 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9885545907182098 = (9.84143620718766 - 0.11263926531189204) / 9.84143620718766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DotBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.04734794448254139 < 4.50002281024412.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 9:01:12 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 127.2974565007275 (T) = (0 -0.02761481451734872) / Math.Sqrt((0.5050460485644405 / (299)) + (0.0005086932771875631 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9948195003998451 = (5.330531155049826 - 0.02761481451734872) / 5.330531155049826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.03356439293195532 < 11.206252546286787.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 106.39792355185382 (T) = (0 -0.08157872999484374) / Math.Sqrt((5.872219864767654 / (299)) + (0.01062230727553541 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9946869751427483 = (15.354479263069434 - 0.08157872999484374) / 15.354479263069434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.230963328293926 < 11.949451655954338.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 138.24113362579956 (T) = (0 -4.177316046526638) / Math.Sqrt((1.1429096488734505 / (299)) + (0.008792238118598356 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.6922880423279202 = (13.57541019247061 - 4.177316046526638) / 13.57541019247061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.InequalityOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 3.476959043844953.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 58.93731786940123 (T) = (0 -0.06544036699074292) / Math.Sqrt((1.2006362369767245 / (299)) + (0.023777020361483932 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9860693256701992 = (4.697573530288618 - 0.06544036699074292) / 4.697573530288618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.EqualsBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.45681352132598074 < 25.609819942629546.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 30.344940905982398 (T) = (0 -0.16312303993446614) / Math.Sqrt((63.832937258678314 / (299)) + (0.03522395132727614 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9885837679376354 = (14.288693418577816 - 0.16312303993446614) / 14.288693418577816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 38.64494511419168.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 290.87599477922765 (T) = (0 -0.10119932605306546) / Math.Sqrt((6.449873933777742 / (299)) + (0.010832847693414584 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9976888346119408 = (43.78714157624503 - 0.10119932605306546) / 43.78714157624503 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.015231366346351524 < 11.174030850208679.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 107.72212240635206 (T) = (0 -0.07133295803595209) / Math.Sqrt((5.7158256664029 / (299)) + (0.010599258449806014 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9953486582373787 = (15.3359958645036 - 0.07133295803595209) / 15.3359958645036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.06134134196148098 < 11.220515466361215.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/10/2023 10:36:48 PM, 5/16/2023 12:30:38 AM falls between 5/7/2023 5:11:33 AM and 5/16/2023 12:30:38 AM.
IsImprovementStdDev: Marked as improvement because 181.66851990683216 (T) = (0 -0.09129557779563449) / Math.Sqrt((1.2046564153019972 / (299)) + (0.009653885496634696 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.9928767841861754 = (12.816623865087662 - 0.09129557779563449) / 12.816623865087662 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository