monolithic lto link time very slow when building with several codegen units

[matthiaskrgr]

I noticed my project https://github.com/matthiaskrgr/cargo-cache took noticeably longer to build with "lto=true" at some point.
I benchmarked link time (touch src/main.rs; cargo build --release) which appear to blow up as soon as we have several codegen units.

link times:

release	stable: 1.23	beta: 1.24	nightly: 1.25
profile.release, lto=true, codegen-units=1	45s	65s	48s
profile.release, lto=true	46s	278s	244s
profile.release (default)	6.3s	6.39	6.43s

So it seem codegen-units=1 makes for bad parallelism while building but speeds up lto because less translation units need to be merged.

Maybe it is possible to dynamically use as little codegen units as possible (but still enough to utilize all cores) when building? //cc #46910

[retep998]

Or maybe just don't use multiple codegen units by default when monolithic LTO is enabled.

[matthiaskrgr]

I think the optimal "balance" for linktime/buildtime with monolithic lto would be to

1. compile all crates with codgen-units=1 as long as we can compile several crates in parallel.
2. if we have to compile a single crate at a time, use ${thread_number} codegen units.

However this will probably introduce indeterministic builds depending on the number of threads a host machine has. So a single core machine would optimize better than a 8 core machine when building the same crate with monolithic lto. :/

[matthiaskrgr]

I made some benchmarks (cpu: i5-2540M dual core), rustc 1.25.0-nightly (b8398d9 2018-02-11)

[profile.release]
lto = true
opt-level = 3
codegen-units = x

codegen-units	total build time
1	7:29
2	7:56
4	7:45
8	8:28
16	10:00

[alexcrichton]

I believe this is fixed in #48163

[alexcrichton]

er, didn't mean to close yet