Significant compile time regressions since Clang 9

[davidbolvansky]

Bugzilla Link	44408
Version	trunk
OS	Linux
Blocks	#44654 #43900
CC	@abrahm,@alinas,@topperc,@dwblaikie,@DougGregor,@fhahn,@zmodem,@RKSimon,@nikic,@zygoloid,@rotateright,@sylvestre,@tstellar,@yuanfang-chen

Extended Description

Recent Phoronix article mentions huge compile time / perf regressions.

For example:
https://www.phoronix.com/scan.php?page=article&item=gcc-clang-3960x&num=4

ImageMagisk - 50 % worse compile time.
PHP - 20 % worse compile time.

[davidbolvansky]

As noted here https://reviews.llvm.org/D68298

That patch is maybe a reason of this slow down.

Hopefully the fix (https://reviews.llvm.org/D72214) will land before 10.0 release.

[davidbolvansky]

https://www.phoronix.com/scan.php?page=news_item&px=AMD-Znver2-Clang-10-Tests

PHP compile time regression is not fixed - 12 % is quite significant regression I believe.

[zmodem]

As noted here https://reviews.llvm.org/D68298

That patch is maybe a reason of this slow down.

Hopefully the fix (https://reviews.llvm.org/D72214) will land before 10.0
release.

Florian, do you know the status here?

[fhahn]

As noted here https://reviews.llvm.org/D68298

That patch is maybe a reason of this slow down.

Hopefully the fix (https://reviews.llvm.org/D72214) will land before 10.0
release.

Florian, do you know the status here?

I've reverted the commit causing the regression on Jan 14: https://reviews.llvm.org/rG192cce10f67e4f22be6d9b8c0975f78ad246d1bd

The patch was causing a compile-time regression, but I am not sure the patch caused the ones mentioned in the article.

I just checked and it looks like it is on the 10.x release branch.

[zmodem]

https://www.phoronix.com/scan.php?page=news_item&px=AMD-Znver2-Clang-10-Tests

PHP compile time regression is not fixed - 12 % is quite significant
regression I believe.

I bisected the PHP regression and ended up at this commit

---

commit 6da79ce
Author: Alina Sbirlea asbirlea@google.com
Date: Wed Sep 4 19:16:04 2019 +0000

[MemorySSA] Re-enable MemorySSA use.

Differential Revision: https://reviews.llvm.org/D58311

llvm-svn: 370957

---

Alina, can you take a look?
It would at least be interesting to know what code is being slow to compile and why.

I bisected like this:

$ cd /work/php
$ wget https://www.php.net/distributions/php-7.4.2.tar.gz
$ tar xf php-7.4.2.tar.gz

And then from my llvm checkout:

$ ninja -C build.release -j1000 clang && ( cd /work/php && rm -rf build && mkdir build && cd build && CC=/work/llvm.monorepo/build.release/bin/clang CXX=/work/llvm.monorepo/build.release/bin/clang++ ../php-7.4.2/configure && time make -j50 )

The build step goes from about 12m25s to 13m27s at this revision.

[davidbolvansky]

Partial explanation (mentions DSE, but it makes sense as a reason for this slowdown) is here:
https://reviews.llvm.org/D72700

"The performance problem's scope goes beyond DSE. The current pass pipeline for both pass managers has a sequence of (gvn, memcpyopt, dse), where all of these use MemDepAnalysis. Switching DSE to MemorySSA may initially get worse compile times, as we need to build both MemDepAnalysis and MemorySSA, but switching all three (use newgvn and port memcpyopt and dse), may be worthwhile. This is the long term goal I have in mind."

I suggest turning off MemorySSA until all passes are ready.

[nikic]

I've opened a related report at bug #​44889, where LoopRotate in particular is affected. Also referenced there is https://perf.rust-lang.org/compare.html?start=9993408439c871c1bc0c0bac3b3008b24039443c&end=52e999afd3be3341463f3b87be02f2bcfb381c17, which shows compile-time regressions in Rust when MemorySSA is enabled. Enabling MemorySSA regresses optimized builds by 1-8%.

(Note that even with MemorySSA disabled, LLVM 10 still regresses opt build compile-times by 4-12%.)

[alinas]

Some clarifications:

• MemorySSA is only enabled for some loop passes (opt flag: -enable-mssa-loop-dependency) and in EarlyCSEwMSSA, which is a FunctionPass. It is not enabled for DSE or other function passes (new gvn is not used). The comment in Significant compile time regressions since Clang 9 #43753 #c6 refers to an ongoing work, nothing is tuned on.

• AFAICT Significant compile time regressions since Clang 9 #43753 #c7 refers to a regression with asserts enabled due to verifications in MemorySSA.

• AFAICT Significant compile time regressions since Clang 9 #43753 #c5 does not appear to be a regression due to verifications. Looking into it, thank you Hans.

[nikic]

Some clarifications:

• MemorySSA is only enabled for some loop passes (opt flag:
  -enable-mssa-loop-dependency) and in EarlyCSEwMSSA, which is a FunctionPass.
  It is not enabled for DSE or other function passes (new gvn is not used).
  The comment in Significant compile time regressions since Clang 9 #43753 #c6 refers to an
  ongoing work, nothing is tuned on.

I believe David's point here was that enabling of MemorySSA in loop passes added 4 extra MemorySSA computations, because it is not being preserved by intermediate passes, so the same issue exists here.

Looking at the pipeline log, it's peculiar that one of the MemorySSA computations happens before a LoopRotate-only loop pass manager, which (as far as I know) does not need MemorySSA, only preserves it -- is it possible to not compute it in that case?

• AFAICT Significant compile time regressions since Clang 9 #43753 #c7 refers to a
  regression with asserts enabled due to verifications in MemorySSA.

As I mentioned on the other report, this regression is with assertions disabled. Regressions with enabled assertions is an unrelated (and thankfully easily fixed) issue.

[alinas]

Understood, here's a preliminary update.

Profiling the test in llvm/llvm-bugzilla-archive#44889 shows there is good amount of time spent in updating dominator tree. I had a pending FIXME which may help address this.
I made a quick prototype to check if this is indeed a relevant cause and for that IR the regression (w/ release-noasserts) went from 9.8% to 6.6% on my machine.
So there is indeed some room to improve there, but the infrastructure change may take a bit to get precisely right.

@​Nikita:
Yes, we can have LoopRotate not run MemorySSA when being in the loop pass manager alone . We have such a mechanism in the new pass manager, but not in the legacy one.

I sent out https://reviews.llvm.org/D74574 which makes LoopRotate preserve MemorySSA when available, but the side effect is also that it splits the loop pass pipeline when LICM and unswitch are run right after.
I'm not seeing a regression on the IR in llvm/llvm-bugzilla-archive#44889 with this, but I'd be happy to get additional timings here.

In order to resolve the loop pass pipeline being split, I'd need to pass the info that the analysis is required in the pass manager, at lib/Transforms/IPO/PassManagerBuilder.cpp:401.

I haven't profiled the php regression yet.

[alinas]

https://reviews.llvm.org/D74574 is now in tree.

I also sent out https://reviews.llvm.org/D74640, which only removes the creation of MemorySSA for LoopRotate when the pass is run alone.

Would you mind testing if these preliminary fixes help?

Thanks!

[zmodem]

I was curious about the ImageMagick regression mentioned in the Phoronix article. I built it like this:

$ wget https://imagemagick.org/download/ImageMagick-7.0.9-24.tar.gz
$ tar xf ImageMagick-7.0.9-24.tar.gz
$ cd ImageMagick-7.0.9-24/
$ mkdir build && cd build
$ CC=/work/llvm.monorepo.rel/build.noasserts/bin/clang CXX=/work/llvm.monorepo.rel/build.noasserts/bin/clang++ ../configure
$ time make -j50

I didn't see any 50% slow-down, but more like 10%. It's not clear exactly what the Phoronix article is measuring though.

Anyway, bisection points to the same MemorySSA enabling commit as the PHP bisection.

I tried building also with the commit and patch in #c11, but it doesn't seem to make any significant difference.

[zmodem]

imagemagick mogrify.c preprocessed
Attaching preprocessed source for one of the ImageMagick files that get a lot slower to build after the MemorySSA enabling.

When building like this:

$ perf stat -r10 /work/llvm.monorepo.rel/build.noasserts/bin/clang -fopenmp -g -O2 -w -mtune=broadwell -fexceptions -pthread -c /tmp/mogrify.i -fPIC -o /dev/null

The build time goes from 7.50s to 10.43s with 6da79ce.

[davidbolvansky]

So.. revert "[MemorySSA] Re-enable MemorySSA use." ?

[davidbolvansky]

I didn't see any 50% slow-down, but more like 10%.

10 % between two commits? Or Clang 9 vs Clang 10? Any data for Clang 9?