Huge performance regression in 2.0.17

[miwelc]

We've experienced a drastic performance regression (between 3x - 8x) in our app by updating from 2.0.16 to 2.017.
On the other hand, size has decreased 14%.
I suspect it's due to LLVM's new pass manager. Is there any flag we can use to select the legacy LLVM pass manager? This new behavior is preventing us from updating as performance is mission critical for our use case and we don't really care about size.
Thanks.

[sbc100]

I think you should be able to just pass -flegacy-pass-manager to the compiler. We should probably have included that in the ChangeLog.

[miwelc]

Would this flag affect system libraries compilation (libc, libc++...)?

[miwelc]

Passing -flegacy-pass-manager at compilation and --lto-legacy-pass-manager at link time doesn't change much.

[sbc100]

Ah yes, the system libraries won't effected by -flegacy-pass-manager on the command line. To fully revert you would need to essentially revert #13818 (at least the emcc.py and building.py parts) then run ./embuilder --clear-cache, then rebuild your project.

[miwelc]

I should have said I was testing on Safari 14.0.3. On Chrome 90 performance doesn't seem to have any regression, regardless of flags.

After further testing on Safari:

• No flags: very bad performance, as in 8x worse.
• -flegacy-pass-manager and --lto-legacy-pass-manager: same very bad performance
• -flegacy-pass-manager, --lto-legacy-pass-manager and -s BINARYEN_EXTRA_PASSES=--one-caller-inline-max-function-size=1: Much better performance but not as good as with 2.0.16
• Only -s BINARYEN_EXTRA_PASSES=--one-caller-inline-max-function-size=1: Same performance as with 2.0.16 😄.

Regardless of flags: ~14% size reduction. Mostly from system libraries?

Seems like the new LLVM pass manager combined with the old Binaryen inlining behavior gives the better performance for our use case on both browsers.

Anyway, I think it would be nice when there are changes to defaults if changelog included the flags needed to revert to old behavior 😅.

[sbc100]

Thats somewhat counter intuitive.. it sounds like the only way to match the old performance is to use the new pass manager.

I guess we may want to revisit the binaryen inlining change and/or file a bug with apple about that.

[miwelc]

I know... I don't get it but I've run each (flag combination x version) multiple times and those are the results I get...
Anyway, I'm going to close this now. Thanks!

[kripken]

Good point about the changelog, thanks, I'll open a PR to update it for both those changes.

[kripken]

As for the perf issue, perhaps more inlining ends up hitting a worse case with Safari register allocation or something like that? If you can provide a testcase that could be interesting to look at. Otherwise let's see if there are more reports of issues.

[mithrendal]

I see performance is dropping when going from 2.0.16 to 2.0.17. it drops from 50fps to 8fps. Strange thing is that it drops only when using Firefox. Chrome and Safari are doing fine on 2.0.17 builds.

code is at https://github.com/dirkwhoffmann/virtualc64web

[sbc100]

Did you try with -s BINARYEN_EXTRA_PASSES=--one-caller-inline-max-function-size=1.. if that solution doesn't work then likely you are no hitting the same issue.

[mithrendal]

thank you @sbc100 that setting brings back the performance of vc64web build with 2.0.17 on firefox to 50fps. Strangely in opposite to @miwelc 's project it does affect only firefox here. Safari runs fine without the setting whereas his project showed the slowdown only on Safari 🤔...