Building easyconfigs with many extensions is much slower with EasyBuild 5.x than with EasyBuild 4.x

[branfosj]

On my system, building R-bundle-CRAN/2025.11-foss-2025b (easybuilders/easybuild-easyconfigs#24465):

• Latest develop (1oth November): 13:44:57
• EB 5.0.0 (pip install easybuild==5.0.0: 10:19:24
• EB 4.9.4 (pip install easybuild==4.9.4): 7:47:08

jsc-zen3 was 17:47:26, so it is not just my system being slow here.

I did a further test with an easyconfig that is the first 49 extensions from R-bundle-CRAN/2025.11-foss-2025:

• EB 4.9.4: 15:08
• EB 5.0.0: 22:27
• EB 5.1.2: 37:47
• EB 5.1.2 with changes from Update fake module for each extension installed #4868 reverted: 25:10

So there is a slowdown from EB 4 -> 5, but the major slowdown is directly from #4868.

We need a better option for resetting the environment between extensions.

R / R-bundle-CRAN / R-bundle-Bioconductor / Perl-bundle-CPAN / Python-bundle-PyPI will benefit enormously on install time.

[Micket]

#5040 for a nicer way to collect profiling data from EB runs, to break down what takes time and where which is very useful here.

[boegel]

There was a suggestion by @Micket to collapse loading of modules (for dependencies) into a single call to Lmod.

That could have pretty big impact in terms of how long it takes to install an extension, I can imagine that preparing the environment often takes longer than the actual installation itself (especially since EasyBuild configures Lmod to ignore its cache...)