[flang][Driver] Enables lto-partitions and fat-lto-object. #158125
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
llvmbot
added
clang
Member
|
@llvm/pr-subscribers-flang-driver @llvm/pr-subscribers-clang Author: Anchu Rajendran S (anchuraj) Changeslto-partition helps in performing parallel lto and fat-lto-objects allows bit code to be embedded in object files generated. Full diff: https://github.com/llvm/llvm-project/pull/158125.diff 8 Files Affected:
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 902a28d60b349..4e19137c91881 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3171,10 +3171,11 @@ def fthin_link_bitcode_EQ : Joined<["-"], "fthin-link-bitcode=">,
MarshallingInfoString<CodeGenOpts<"ThinLinkBitcodeFile">>;
defm fat_lto_objects : BoolFOption<"fat-lto-objects",
CodeGenOpts<"FatLTO">, DefaultFalse,
- PosFlag<SetTrue, [], [ClangOption, CC1Option], "Enable">,
- NegFlag<SetFalse, [], [ClangOption, CC1Option], "Disable">,
+ PosFlag<SetTrue, [], [ClangOption, CC1Option, FlangOption, FC1Option], "Enable">,
+ NegFlag<SetFalse, [], [ClangOption, CC1Option, FlangOption, FC1Option], "Disable">,
BothFlags<[], [ClangOption, CC1Option], " fat LTO object support">>;
def flto_partitions_EQ : Joined<["-"], "flto-partitions=">, Group<f_Group>,
+ Visibility<[ClangOption, FlangOption, CC1Option, FC1Option]>,
HelpText<"Number of partitions to use for parallel full LTO codegen, ld.lld only.">;
def fmacro_backtrace_limit_EQ : Joined<["-"], "fmacro-backtrace-limit=">,
Group<f_Group>, Visibility<[ClangOption, CC1Option, CLOption]>,
diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp
index 1535f4cebf436..f14a654cedaa3 100644
--- a/clang/lib/Driver/ToolChains/Flang.cpp
+++ b/clang/lib/Driver/ToolChains/Flang.cpp
@@ -182,6 +182,29 @@ void Flang::addCodegenOptions(const ArgList &Args,
CmdArgs.push_back("-fcoarray");
}
+void Flang::addLTOOptions(const ArgList &Args, ArgStringList &CmdArgs) const {
+ const auto &TC = getToolChain();
+ const Driver &D = TC.getDriver();
+ DiagnosticsEngine &Diags = D.getDiags();
+ LTOKind LTOMode = D.getLTOMode();
+ // LTO mode is parsed by the Clang driver library.
+ assert(LTOMode != LTOK_Unknown && "Unknown LTO mode.");
+ if (LTOMode == LTOK_Full)
+ CmdArgs.push_back("-flto=full");
+ else if (LTOMode == LTOK_Thin) {
+ Diags.Report(
+ Diags.getCustomDiagID(DiagnosticsEngine::Warning,
+ "the option '-flto=thin' is a work in progress"));
+ CmdArgs.push_back("-flto=thin");
+ }
+ if (Args.hasArg(options::OPT_flto_partitions_EQ)) {
+ StringRef A = Args.getLastArgValue(options::OPT_flto_partitions_EQ, "8");
+ CmdArgs.push_back(Args.MakeArgString("-flto-partitions=" + A));
+ }
+ Args.addAllArgs(CmdArgs, {options::OPT_ffat_lto_objects,
+ options::OPT_fno_fat_lto_objects});
+}
+
void Flang::addPicOptions(const ArgList &Args, ArgStringList &CmdArgs) const {
// ParsePICArgs parses -fPIC/-fPIE and their variants and returns a tuple of
// (RelocationModel, PICLevel, IsPIE).
@@ -884,17 +907,7 @@ void Flang::ConstructJob(Compilation &C, const JobAction &JA,
handleColorDiagnosticsArgs(D, Args, CmdArgs);
- // LTO mode is parsed by the Clang driver library.
- LTOKind LTOMode = D.getLTOMode();
- assert(LTOMode != LTOK_Unknown && "Unknown LTO mode.");
- if (LTOMode == LTOK_Full)
- CmdArgs.push_back("-flto=full");
- else if (LTOMode == LTOK_Thin) {
- Diags.Report(
- Diags.getCustomDiagID(DiagnosticsEngine::Warning,
- "the option '-flto=thin' is a work in progress"));
- CmdArgs.push_back("-flto=thin");
- }
+ addLTOOptions(Args, CmdArgs);
// -fPIC and related options.
addPicOptions(Args, CmdArgs);
diff --git a/clang/lib/Driver/ToolChains/Flang.h b/clang/lib/Driver/ToolChains/Flang.h
index 7c24a623af393..98167e1b75e15 100644
--- a/clang/lib/Driver/ToolChains/Flang.h
+++ b/clang/lib/Driver/ToolChains/Flang.h
@@ -40,6 +40,14 @@ class LLVM_LIBRARY_VISIBILITY Flang : public Tool {
void addPreprocessingOptions(const llvm::opt::ArgList &Args,
llvm::opt::ArgStringList &CmdArgs) const;
+ /// Extract LTO options from the driver arguments and add them to
+ /// the command arguments.
+ ///
+ /// \param [in] Args The list of input driver arguments
+ /// \param [out] CmdArgs The list of output command arguments
+ void addLTOOptions(const llvm::opt::ArgList &Args,
+ llvm::opt::ArgStringList &CmdArgs) const;
+
/// Extract PIC options from the driver arguments and add them to
/// the command arguments.
///
diff --git a/flang/include/flang/Frontend/CodeGenOptions.def b/flang/include/flang/Frontend/CodeGenOptions.def
index cdeea93c9aecb..fa29b8ed79532 100644
--- a/flang/include/flang/Frontend/CodeGenOptions.def
+++ b/flang/include/flang/Frontend/CodeGenOptions.def
@@ -35,6 +35,7 @@ CODEGENOPT(InstrumentFunctions, 1, 0) ///< Set when -finstrument_functions is
CODEGENOPT(IsPIE, 1, 0) ///< PIE level is the same as PIC Level.
CODEGENOPT(PICLevel, 2, 0) ///< PIC level of the LLVM module.
+CODEGENOPT(PrepareForFatLTO , 1, 0) ///< Set when -ffat-lto-objects is enabled.
CODEGENOPT(PrepareForFullLTO , 1, 0) ///< Set when -flto is enabled on the
///< compile step.
CODEGENOPT(PrepareForThinLTO , 1, 0) ///< Set when -flto=thin is enabled on the
diff --git a/flang/lib/Frontend/CompilerInvocation.cpp b/flang/lib/Frontend/CompilerInvocation.cpp
index fb3a132cae30e..6cca1337d4333 100644
--- a/flang/lib/Frontend/CompilerInvocation.cpp
+++ b/flang/lib/Frontend/CompilerInvocation.cpp
@@ -325,6 +325,10 @@ static void parseCodeGenArgs(Fortran::frontend::CodeGenOptions &opts,
if (args.hasArg(clang::driver::options::OPT_finstrument_functions))
opts.InstrumentFunctions = 1;
+ if (args.hasArg(clang::driver::options::OPT_ffat_lto_objects)) {
+ opts.PrepareForFatLTO = true;
+ }
+
// -flto=full/thin option.
if (const llvm::opt::Arg *a =
args.getLastArg(clang::driver::options::OPT_flto_EQ)) {
diff --git a/flang/lib/Frontend/FrontendActions.cpp b/flang/lib/Frontend/FrontendActions.cpp
index 3bef6b1c31825..bcce9f6ede62e 100644
--- a/flang/lib/Frontend/FrontendActions.cpp
+++ b/flang/lib/Frontend/FrontendActions.cpp
@@ -995,7 +995,9 @@ void CodeGenAction::runOptimizationPipeline(llvm::raw_pwrite_stream &os) {
// Create the pass manager.
llvm::ModulePassManager mpm;
- if (opts.PrepareForFullLTO)
+ if (opts.PrepareForFatLTO)
+ mpm = pb.buildFatLTODefaultPipeline(level, opts.PrepareForThinLTO, true);
+ else if (opts.PrepareForFullLTO)
mpm = pb.buildLTOPreLinkDefaultPipeline(level);
else if (opts.PrepareForThinLTO)
mpm = pb.buildThinLTOPreLinkDefaultPipeline(level);
diff --git a/flang/test/Driver/lto-fatlto.f90 b/flang/test/Driver/lto-fatlto.f90
new file mode 100644
index 0000000000000..4602cf8f38890
--- /dev/null
+++ b/flang/test/Driver/lto-fatlto.f90
@@ -0,0 +1,20 @@
+! REQUIRES: x86-registered-target
+! checks fatlto objects: that valid bitcode is included in the object file generated.
+
+! RUN: %flang -fc1 -triple x86_64-unknown-linux-gnu -flto -ffat-lto-objects -emit-obj %s -o %t.o
+! RUN: llvm-readelf -S %t.o | FileCheck %s --check-prefixes=ELF
+! RUN: llvm-objcopy --dump-section=.llvm.lto=%t.bc %t.o
+! RUN: llvm-dis %t.bc -o - | FileCheck %s --check-prefixes=DIS
+
+! ELF: .llvm.lto
+! DIS: define void @_QQmain()
+! DIS-NEXT: ret void
+! DIS-NEXT: }
+
+! RUN: %flang -fc1 -triple x86_64-unknown-linux-gnu -flto -ffat-lto-objects -S %s -o - | FileCheck %s --check-prefixes=ASM
+
+! ASM: .section .llvm.lto,"e",@llvm_lto
+! ASM-NEXT: .Lllvm.embedded.object:
+! ASM-NEXT: .asciz "BC
+! ASM-NEXT: .size .Lllvm.embedded.object
+end program
diff --git a/flang/test/Driver/lto-lld-flags.f90 b/flang/test/Driver/lto-lld-flags.f90
new file mode 100644
index 0000000000000..4176e27b89d42
--- /dev/null
+++ b/flang/test/Driver/lto-lld-flags.f90
@@ -0,0 +1,18 @@
+! check flto-partitions is passed to lld, fc1
+! RUN: %flang -### -fuse-ld=lld -flto=full -flto-partitions=16 %s 2>&1 | FileCheck %s --check-prefixes=LLD-PART,FC1-PART
+
+! FC1-PART: "-fc1"
+! FC1-PART-SAME: "-flto=full"
+! FC1-PART-SAME: "-flto-partitions=16"
+! LLD-PART: ld.lld
+! LLD-PART-SAME: "--lto-partitions=16"
+
+! check fat-lto-objects is passed to lld, fc1
+! RUN: %flang -### -fuse-ld=lld -flto -ffat-lto-objects %s 2>&1 | FileCheck %s --check-prefixes=LLD-FAT,FC1-FAT
+
+! FC1-FAT: "-fc1"
+! FC1-FAT-SAME: "-flto=full"
+! FC1-FAT-SAME: "-ffat-lto-objects"
+! LLD-FAT: ld.lld
+! LLD-FAT-SAME: "--fat-lto-objects"
+end program
\ No newline at end of file
|
anchuraj
requested review from
kiranchandramohan,
mjklemm,
raghavendhra,
skatrak,
tarunprabhu and
tblah
anchuraj
force-pushed
the
lto-flags
branch
4 times, most recently
from
128bb9f to
52c3055
Compare
anchuraj
changed the title
lto-partition helps in performing parallel lto and fat-lto-objects allows bit code to be embedded in object files generated.
tarunprabhu
requested changes
Sep 15, 2025
tarunprabhu
reviewed
Sep 17, 2025
Contributor
tarunprabhu
left a comment
tarunprabhu
left a comment
There was a problem hiding this comment.
Thanks. Looks like I missed a couple of things the first time around. Just some clarifications.
Contributor
There was a problem hiding this comment.
I know that this isn't part of this PR, but do you know what flto=thin does in flang? Does it do anything at all or do we just accept this option, then ignore it? This warning is not very informative, and I wonder if it is worth providing something different.
Contributor
Author
There was a problem hiding this comment.
https://reviews.llvm.org/D142420 is the changes I can see. It adds the pipeline buildThinLTOPreLinkDefaultPipeline(level) in FrontendAction.cpp. It does not ignore the option but says further development is required.
Contributor
Author
There was a problem hiding this comment.
Thank you for bringing this up. There are TODOs mentioned in the above PR:
The TODOs would be:
Add support for -f[no]split-lto-unit
Emit LTO summary
Use ThinLTOBitcodeWriterPass for thin lto
Contributor
There was a problem hiding this comment.
Thanks for looking into this. It looks like there were some concerns about that commit message even back then. Did you have plans to work on any of these?
Contributor
Author
There was a problem hiding this comment.
I plan to work on enabling module summary.
abidh
reviewed
Sep 17, 2025
Contributor
abidh
left a comment
abidh
left a comment
There was a problem hiding this comment.
Thanks for the PR. Apart from a few minor nits, it looks good to me.
flang/test/Driver/lto-lld-flags.f90
Outdated
Contributor
There was a problem hiding this comment.
No new line at the end of the file.
Contributor
There was a problem hiding this comment.
May be add a comment why this llvm::Triple::Apple check is required.
Contributor
Author
There was a problem hiding this comment.
Thank you! Updated.
Contributor
There was a problem hiding this comment.
Like other places in the PR, you may want to add && "Unknown LTO mode" here too.
Contributor
There was a problem hiding this comment.
I think we can remove the Braces for this if.
anchuraj
force-pushed
the
lto-flags
branch
2 times, most recently
from
3375fc4 to
7034c46
Compare
tarunprabhu
approved these changes
Sep 17, 2025
Contributor
tarunprabhu
left a comment
tarunprabhu
left a comment
There was a problem hiding this comment.
Thanks for the changes and investigations. LGTM, but wait for @abidh to take a look.
abidh
approved these changes
Sep 18, 2025
Contributor
abidh
left a comment
abidh
left a comment
There was a problem hiding this comment.
Thanks for the changes. LGTM.
anchuraj
added a commit
that referenced
this pull request
Sep 18, 2025
#158125 resulted in CI failure as `llvm-readelf` and `llvm-objcopy` were not listed in flang test deps. This PR fixes it.
llvm-sync bot
pushed a commit
to arm/arm-toolchain
that referenced
this pull request
Sep 18, 2025
llvm/llvm-project#158125 resulted in CI failure as `llvm-readelf` and `llvm-objcopy` were not listed in flang test deps. This PR fixes it.
Contributor
|
I'm seeing failures downstream in |
Contributor
|
|
Contributor
|
This change also broke https://lab.llvm.org/buildbot/#/builders/130/builds/15415. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
flto-partition helps in performing parallel lto and ffat-lto-objects allows bit code to be embedded in object files generated. This PR enables the support of these flags in flang.