-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 375cc6d80d3620415309c35d4dbcfc2172f50fab |
| Compare | 231f85f754110bea22930effcbd348babec32423 |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Improvements in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LastIndexOfAnyValues - Duration of single invocation | 319.73 ns | 229.89 ns | 0.72 | 0.01 | False | |||||
| IndexOfAnyThreeValues - Duration of single invocation | 377.22 ns | 245.79 ns | 0.65 | 0.01 | False | |||||
| IndexOfAnyThreeValues - Duration of single invocation | 172.06 ns | 36.95 ns | 0.21 | 0.10 | False | |||||
| IndexOfAnyFourValues - Duration of single invocation | 200.14 ns | 46.39 ns | 0.23 | 0.03 | False | |||||
| IndexOfAnyFourValues - Duration of single invocation | 413.12 ns | 265.26 ns | 0.64 | 0.02 | False | |||||
| IndexOfAnyFiveValues - Duration of single invocation | 244.86 ns | 49.61 ns | 0.20 | 0.03 | False | |||||
| IndexOfAnyTwoValues - Duration of single invocation | 325.26 ns | 231.56 ns | 0.71 | 0.01 | True | |||||
| IndexOfAnyFiveValues - Duration of single invocation | 469.29 ns | 275.80 ns | 0.59 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to
runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - AOT MONO build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create aot directory
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span<Byte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Memory.Span<Byte>*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Memory.Span<Byte>* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200Payloads
Histogram
System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 229.8941003006807 < 303.5356693064128.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 69.13994644575584 (T) = (0 -243.2133788927007) / Math.Sqrt((385.55641949970914 / (299)) + (19.714755895131535 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3329084381267554 = (364.58770098925964 - 243.2133788927007) / 364.58770098925964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 245.79097140021463 < 361.4900605959518.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 103.0466448427871 (T) = (0 -256.07722466927913) / Math.Sqrt((633.4273218265018 / (299)) + (12.155354738129962 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4194429354351725 = (441.08880986786176 - 256.07722466927913) / 441.08880986786176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.94629677692893 < 164.37494939773305.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 4.07336275204696 (T) = (0 -47.83974733506966) / Math.Sqrt((2297.8068534847066 / (299)) + (13.425339495218973 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.20261083801861668 = (59.99548227642775 - 47.83974733506966) / 59.99548227642775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 46.39140844953307 < 189.8009363045018.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 5.704924889883067 (T) = (0 -57.35590554894207) / Math.Sqrt((2752.066335702467 / (299)) + (14.485991751473478 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.24393002292517704 = (75.86057810528027 - 57.35590554894207) / 75.86057810528027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 265.26305786280227 < 394.67898261123435.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 74.14620492077586 (T) = (0 -281.83087989920085) / Math.Sqrt((2866.929436416672 / (299)) + (33.92041536419218 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4836138123272347 = (545.7754034230627 - 281.83087989920085) / 545.7754034230627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 33)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.60938748039751 < 232.9924069204933.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 7.687061545439139 (T) = (0 -59.31864330458853) / Math.Sqrt((4186.118290353213 / (299)) + (11.058978444994272 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3342181644679864 = (89.09621761787497 - 59.31864330458853) / 89.09621761787497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 231.56332384328616 < 308.2989627387257.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 49.589064782804556 (T) = (0 -246.46719842825314) / Math.Sqrt((10.662926149611096 / (299)) + (32.41598727744127 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2578705604937988 = (332.1081004309002 - 246.46719842825314) / 332.1081004309002 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 275.80414869763075 < 447.3904949230281.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 82.3431092499745 (T) = (0 -286.887047022449) / Math.Sqrt((5663.59847250277 / (299)) + (14.534954271955806 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.5637040155008389 = (657.5514265889391 - 286.887047022449) / 657.5514265889391 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 375cc6d80d3620415309c35d4dbcfc2172f50fab |
| Compare | 231f85f754110bea22930effcbd348babec32423 |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsSuffix_DifferentLastChar - Duration of single invocation | 3.41 μs | 741.57 ns | 0.22 | 0.01 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 1.72 μs | 373.42 ns | 0.22 | 0.01 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 1.72 μs | 370.54 ns | 0.22 | 0.01 | False | |||||
| IsPrefix_DifferentFirstChar - Duration of single invocation | 145.38 ns | 44.13 ns | 0.30 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to
runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - AOT MONO build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create aot directory
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Globalization.Tests.StringSearch* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.Globalization.Tests.StringSearch*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.Globalization.Tests.StringSearch* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200Payloads
Histogram
System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 741.5660266117162 < 3.2415549677045945.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 12.380050627161038 (T) = (0 -737.5672800779557) / Math.Sqrt((740189.4214497829 / (299)) + (6.09624124178183 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.455108679926189 = (1353.6043847753801 - 737.5672800779557) / 1353.6043847753801 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 373.423408557272 < 1.6341837012368605.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 12.708716712407265 (T) = (0 -368.6515890668631) / Math.Sqrt((191735.35173415157 / (299)) + (3.13226486393792 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4661448737813825 = (690.5461256466377 - 368.6515890668631) / 690.5461256466377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 370.5354211834059 < 1.636571540178377.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 12.591508687414894 (T) = (0 -372.4331009242467) / Math.Sqrt((190842.94927576478 / (299)) + (8.567519160242663 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.4608195620225443 = (690.7392677696124 - 372.4331009242467) / 690.7392677696124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 44.12994447926596 < 138.38699941443727.
IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 4.757160746682251 (T) = (0 -44.62298655938136) / Math.Sqrt((1526.540536895878 / (299)) + (1.2249665974629802 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19581618514631538 = (55.48854097181773 - 44.62298655938136) / 55.48854097181773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 375cc6d80d3620415309c35d4dbcfc2172f50fab |
| Compare | 231f85f754110bea22930effcbd348babec32423 |
| Diff | Diff |
| Configs | CompilationMode:tiered, LLVM:false, MonoAOT:true, MonoInterpreter:false, RunKind:micro_mono |
Improvements in System.IO.Tests.BinaryWriterExtendedTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteAsciiCharArray - Duration of single invocation | 5.45 ms | 4.00 ms | 0.73 | 0.02 | False | |||||
| WriteAsciiString - Duration of single invocation | 19.80 μs | 14.21 μs | 0.72 | 0.01 | False | |||||
| WriteAsciiCharArray - Duration of single invocation | 19.95 μs | 14.55 μs | 0.73 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Repro Steps
Prerequisites (Files either built locally (with build.(sh/cmd) or downloaded from payload above (if same system setup) (in this order))
- Libraries build extracted to
runtime/artifactsor build instructions: Libraries README args:-subset libs+libs.tests -rc release -configuration Release -arch $RunArch -framework net8.0 - CoreCLR product build extracted to
runtime/artifacts/bin/coreclr/$RunOS.$RunArch.Release, build instructions: CoreCLR README args:-subset clr+libs -rc release -configuration Release -arch $RunArch -framework net8.0 - AOT MONO build extracted to
runtime/artifacts/bin/mono/$RunOS.$RunArch.Release, build instructions: MONO README args:-arch $RunArch -os $RunOS -s mono+libs+host+packs -c Release /p:CrossBuild=false /p:MonoLLVMUseCxx11Abi=false - Dotnet SDK installed for dotnet commands
- Running commands from the runtime folder
Linux
# Set $RunDir to the runtime directory
RunDir=`pwd`
# Set the OS, arch, and OSId
RunOS='linux'
RunOSId='linux'
RunArch='x64'
# Create aot directory
mkdir -p $RunDir/artifacts/bin/aot/sgen
mkdir -p $RunDir/artifacts/bin/aot/pack
cp -r $RunDir/artifacts/obj/mono/$RunOS.$RunArch.Release/mono/* $RunDir/artifacts/bin/aot/sgen
cp -r $RunDir/artifacts/bin/microsoft.netcore.app.runtime.$RunOS-$RunArch/Release/* $RunDir/artifacts/bin/aot/pack
# Create Core Root
$RunDir/src/tests/build.sh release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir/performance
# One line run:
python3 $RunDir/performance/scripts/benchmarks_ci.py --csproj $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --packages $RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir/performance/artifacts/packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryWriterExtendedTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir/artifacts/bin/aot/sgen/mini/mono-sgen --customruntimepack $RunDir/artifacts/bin/aot/pack --aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir/artifacts/BenchmarkDotNet.Artifacts --packages $RunDir/performance/artifacts/packages --buildTimeout 1200Windows
# Set $RunDir to the runtime directory
$RunDir="FullPathHere"
# Set the OS, arch, and OSId
RunOS='windows'
RunOSId='win'
RunArch='x64'
# Create aot directory
mkdir $RunDir\artifacts\bin\aot\sgen
mkdir $RunDir\artifacts\bin\aot\pack
xcopy $RunDir\artifacts\obj\mono\$RunOS.$RunArch.Release\mono $RunDir\artifacts\bin\aot\sgen\ /e /y
xcopy $RunDir\artifacts\bin\microsoft.netcore.app.runtime.$RunOSId-$RunArch\Release $RunDir\artifacts\bin\aot\pack\ /e /y
# Create Core Root
$RunDir\src\tests\build.cmd release $RunArch generatelayoutonly /p:LibrariesConfiguration=Release
# Clone performance
git clone --branch main --depth 1 --quiet https://github.com/dotnet/performance.git $RunDir\performance
# One line run:
python3 $RunDir\performance\scripts\benchmarks_ci.py --csproj $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --incremental no --architecture $RunArch -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*' --bdn-artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --bdn-arguments="--anyCategories Libraries Runtime --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack --aotcompilermode llvm --logBuildOutput --generateBinLog"
# Individual Commands:
# Restore
dotnet restore $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --packages $RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Build
dotnet build $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore /p:NuGetPackageRoot=$RunDir\performance\artifacts\packages /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
# Run
dotnet run --project $RunDir\performance\src\benchmarks\micro\MicroBenchmarks.csproj --configuration Release --framework net8.0 --no-restore --no-build -- --filter System.IO.Tests.BinaryWriterExtendedTests* --anyCategories Libraries Runtime " --category-exclusion-filter NoAOT NoWASM --runtimes monoaotllvm --aotcompilerpath $RunDir\artifacts\bin\aot\sgen\mini\mono-sgen.exe --customruntimepack $RunDir\artifacts\bin\aot\pack -aotcompilermode llvm --logBuildOutput --generateBinLog " --artifacts $RunDir\artifacts\BenchmarkDotNet.Artifacts --packages $RunDir\performance\artifacts\packages --buildTimeout 1200Payloads
Histogram
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.002280017708333 < 5.155423950138889.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/17/2023 5:51:51 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 16.052410912209506 (T) = (0 -3966539.9361363635) / Math.Sqrt((518696192758.4102 / (299)) + (7695682103.345787 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.1664404933455168 = (4758556.413154226 - 3966539.9361363635) / 4758556.413154226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.209807684178744 < 18.820871694384792.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/17/2023 5:51:51 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 24.967826869639488 (T) = (0 -14408.138873037835) / Math.Sqrt((6843599.189599262 / (299)) + (30240.90399100656 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2172005450182379 = (18405.91326596353 - 14408.138873037835) / 18405.91326596353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.547142639931208 < 18.903306558009778.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 4/12/2023 6:21:10 AM, 4/17/2023 5:51:51 AM, 4/28/2023 1:33:49 PM, 5/2/2023 12:51:17 PM falls between 4/23/2023 8:38:14 PM and 5/2/2023 12:51:17 PM.
IsImprovementStdDev: Marked as improvement because 27.27369675930641 (T) = (0 -14604.856836707737) / Math.Sqrt((5283441.7565725045 / (299)) + (11022.732675098126 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.2033017463398939 = (18331.729446640133 - 14604.856836707737) / 18331.729446640133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository





