Skip to content

Conversation

@T-Gro
Copy link
Member

@T-Gro T-Gro commented Oct 12, 2022

This benchmark suite now proves that F# is not very clever in incremental builds - any change of files invalidates the subsequent files, even if their signatures would not change.

It can be used as a basis for testing improvement via new features/ideas.

The current results are:

BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.22000.1098/21H2)
11th Gen Intel Core i9-11950H 2.60GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100-rc.2.22477.23
  [Host]  : .NET 7.0.0 (7.0.22.47203), X64 RyuJIT AVX2 DEBUG
  LongRun : .NET 7.0.0 (7.0.22.47203), X64 RyuJIT AVX2

Job=LongRun  IterationCount=100  LaunchCount=3  
WarmupCount=15  
Method Mean Error StdDev Median Ratio RatioSD Rank Gen0 Gen1 Gen2 Allocated Alloc Ratio
ParseProjectAsIs 98.19 ns 0.723 ns 3.766 ns 98.70 ns 0.001 0.00 I 0.0223 - - 280 B 0.17
ParseProjectWithFullCacheClear 89,898.93 ns 190.661 ns 986.896 ns 89,727.05 ns 1.000 0.00 II 1000.0000 1000.0000 1000.0000 1664 B 1.00
ParseProjectWithChangingFirstFile 531,555.27 ns 2,579.920 ns 13,145.366 ns 531,244.04 ns 5.914 0.16 III - - - 8600 B 5.17
ParseProjectWithChangingMiddleFile 544,790.07 ns 7,099.963 ns 36,750.697 ns 553,684.03 ns 6.061 0.41 IV - - - 8896 B 5.35
ParseProjectWithChanging25thPercentileFile 553,487.15 ns 7,892.799 ns 40,713.491 ns 551,352.78 ns 6.158 0.46 IV - - - 8896 B 5.35

@vzarytovskii
Copy link
Member

Great! Might be good to eventually include it in dotnet/performance too.

Copy link
Contributor

@psfinaki psfinaki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This benchmark suite now proves that F# is not very clever in incremental builds

the-truth-hits-hard-hard-pills-to-swallow

@T-Gro T-Gro merged commit 434391e into main Oct 13, 2022
@T-Gro T-Gro deleted the feature/project-with-cascading-files-benchmarks branch October 13, 2022 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants