Revalidate perf of batch/server GC settings for fsc, fsi on .NET Framework and .NET Core

[dsyme]

This is a tracking issue to make sure we revalidate the settings that we use for GC on both .NET Framework and .NET 6/7.

• gcServer
• LatencyMode.Batch

So the current situation is

• .NET SDK compiler: Server off, Batch on
• .NET Framework compiler: Server on, Batch on

[dsyme]

Initially I thought ServerGC being slower as --times is showing a longer for overall total CPU time with it on. However overall wall-clock time is lower.

To see this, I took logs using MSBuild -verbosity d which includes a processor graph at the end with actual times for each process launched by the build and also shows core utilization. According to --times, turning Server GC off takes total CPU time consumed by F# compiler in the whole build from ~400s to ~300s. But the actual end-to-end time goes up by 10% or so.

So don't rely on --times. End-to-end wallclock times now look more reliable.

This means we should turn ON server GC for the compiler.

[dsyme]

In more detail, I did some more validation on whether Server GC is a gain or a loss.

In #13739 I had validated manually that turning gcServer off improved compile time for compiling FSharp.Compiler.Service.dll alone. However tried to validate a perf improvement on end-to-end build of the repo (when the Bootstrap compiler uses .NET Framework). Surprisingly I didn't see this

End-to-end build timings for .\build -c Release

• With gcServer: 139s
• Without gcServer: 146s

This surprised me as the overall compile time is quite heavily dominated by compiling FSharp.Compiler.Service.dll. Digging deeper into the logs for the end to end build, --times indeed reports the same as I saw - that Server GC is slower for FSharp.Compiler.Service.dll:

FSharp.Compiler.Service.dll:
    With Server GC:            TIME: 85.0 Delta:  9.1 Mem: 571 G0:  12 G1:  7 G2:  3 [Write .NET Binary]
    Without Server GC:         TIME: 61.0 Delta:  4.0 Mem: 784 G0: 121 G1: 107 G2:  0 [Write .NET Binary]

Adding --times to all compilations and analysing the entire build log reveals we indeed spend much much less CPU time in the F# compiler overall when Server GC is off:

grep "Write .NET Binary" /mnt/c/GitHub/dsyme/with.log | col2 | paste -sd+ - | bc
428.7
grep "Write .NET Binary" /mnt/c/GitHub/dsyme/without.log | col2 | paste -sd+ - | bc
320.2

This is a huge difference - so I was really confused why this doesn't translate to better end-to-end build times. The memory usage reported by --times is also reasonable:

grep "Write .NET Binary" /mnt/c/GitHub/dsyme/with.log | col6 | paste -sd+ - | bc
10972
grep "Write .NET Binary" /mnt/c/GitHub/dsyme/without.log | col6 | paste -sd+ - | bc
12515

Then I looked at the actual end-to-end process times and CPU core utilizations reported by .\build -verbosity d. This showed that the actual wall-clock execution times for processes were substantially lower with Server GC, to take one example:

With:             x       x       x       x       x       |       x       |       |       x       |       x       x       x       x       x       x       |       x       x        23.716s     106.233s ++++ (scale 1:100)

Without:          |       x       x       x       x       |       x       |       x       x       x       x       x       |       |       x       |       x       |       647      31.264s     115.905s ++++++ (scale 1:100)

Here the "with server GC" is using 15 cores and is 8% faster, while "without server GC" is using 13 cores

So, in short, don't trust --times which reports Process.GetCurrentProcess().UserProcessorTime.TotalSeconds and instead look at either wall-clock or these individual process reports. I'll adjust --times to report wall-clock time.

[dsyme]

Here's our latest main compiler running on .NET 6, compiling FSharp.Compiler.Service.dll, batch mode turned off.

Without server GC,    Total CPU = 56.2     Wallclock = 50.4
With    server GC,    Total CPU = 78.6     Wallclock = 45.1

So wallclock gain is 10% with Server GC.

[dsyme]

Closing as done. We need to enable Server mode by default, and have removed use of Batch