Skip to content

[DTLTO][Clang] Add support for Integrated Distributed ThinLTO #147265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jul 15, 2025
Merged

[DTLTO][Clang] Add support for Integrated Distributed ThinLTO #147265

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
737bb8c
[DTLTO][Clang] Add support for Integrated Distributed ThinLTO
bd1976bris Jun 5, 2025
af66910
Avoid committing to any work on Sony’s behalf that’s beyond my remit.
bd1976bris Jul 10, 2025
fd51379
shorten argument strings for readability
bd1976bris Jul 10, 2025
f2c6b8d
Use meaningful prefixes for all check lines.
bd1976bris Jul 10, 2025
cdce7ff
NOMORE -> --implicit-check-not
bd1976bris Jul 10, 2025
75289e7
Use shared response files for all FileCheck invocations.
bd1976bris Jul 10, 2025
bb2f7d3
Simplify test by including more arguments in linker shared response file
bd1976bris Jul 10, 2025
00321cf
Improve match for clang executable
bd1976bris Jul 10, 2025
fb25cda
Removed unneeded -Werror as we can rely on the implicit check not check
bd1976bris Jul 10, 2025
44abe76
Introduce a second linker response file which will simplify the test …
bd1976bris Jul 10, 2025
820a50d
Move option handling into addLTOOptions for consistency.
bd1976bris Jul 10, 2025
60e87c9
On linux the clang exectuable might be clang-20. Fix match.
bd1976bris Jul 10, 2025
c961cc0
Use %/s consistently in response files.
bd1976bris Jul 11, 2025
93dda6f
Use -Werror and remove FileCheck response file
bd1976bris Jul 14, 2025
c8e8305
Remove other response file usage
bd1976bris Jul 14, 2025
1b0257c
Replace accidentally dropped implicit check not lines
bd1976bris Jul 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion clang/docs/ThinLTO.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ If ``-fthinlto-distributor=`` is specified, Clang supplies the path to a
compiler to be executed remotely to perform the ThinLTO backend
compilations. Currently, this is Clang itself.

Note that currently, DTLTO is only supported in some LLD flavors. Support will
Note that currently, DTLTO is only supported in some LLD flavors. Support can
be added to other LLD flavours in the future.
See `DTLTO <https://lld.llvm.org/dtlto.html>`_ for more information.

Expand Down
11 changes: 11 additions & 0 deletions clang/lib/Driver/ToolChains/CommonArgs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1309,6 +1309,17 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
if (Args.hasArg(options::OPT_ftime_report))
CmdArgs.push_back(
Args.MakeArgString(Twine(PluginOptPrefix) + "-time-passes"));

if (Arg *A = Args.getLastArg(options::OPT_fthinlto_distributor_EQ)) {
CmdArgs.push_back(
Args.MakeArgString("--thinlto-distributor=" + Twine(A->getValue())));
CmdArgs.push_back(
Args.MakeArgString("--thinlto-remote-compiler=" +
Twine(ToolChain.getDriver().getClangProgramPath())));

for (auto A : Args.getAllArgValues(options::OPT_Xthinlto_distributor_EQ))
CmdArgs.push_back(Args.MakeArgString("--thinlto-distributor-arg=" + A));
}
}

void tools::addOpenMPRuntimeLibraryPath(const ToolChain &TC,
Expand Down
15 changes: 0 additions & 15 deletions clang/lib/Driver/ToolChains/Gnu.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,21 +455,6 @@ void tools::gnutools::Linker::ConstructJob(Compilation &C, const JobAction &JA,
addLTOOptions(ToolChain, Args, CmdArgs, Output, Inputs,
D.getLTOMode() == LTOK_Thin);

// Forward the DTLTO options to the linker. We add these unconditionally,
// rather than in addLTOOptions() as it is the linker that decides whether to
// do LTO or not dependent upon whether there are any bitcode input files in
// the link.
if (Arg *A = Args.getLastArg(options::OPT_fthinlto_distributor_EQ)) {
CmdArgs.push_back(
Args.MakeArgString("--thinlto-distributor=" + Twine(A->getValue())));
CmdArgs.push_back(
Args.MakeArgString("--thinlto-remote-compiler=" +
Twine(ToolChain.getDriver().getClangProgramPath())));

for (auto A : Args.getAllArgValues(options::OPT_Xthinlto_distributor_EQ))
CmdArgs.push_back(Args.MakeArgString("--thinlto-distributor-arg=" + A));
}

if (Args.hasArg(options::OPT_Z_Xlinker__no_demangle))
CmdArgs.push_back("--no-demangle");

Expand Down
65 changes: 35 additions & 30 deletions clang/test/Driver/DTLTO/dtlto.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,42 +2,47 @@

/// Check DTLTO options are forwarded to the linker.

// RUN: echo "--target=x86_64-linux-gnu \
// RUN: -Xthinlto-distributor=distarg1 \
// RUN: -Xthinlto-distributor=distarg2,distarg3 \
// RUN: -fuse-ld=lld" > %t.rsp

/// Check that options are forwarded as expected with --thinlto-distributor=.
// RUN: %clang -### @%t.rsp -fthinlto-distributor=dist.exe %s 2>&1 | \
// RUN: FileCheck %s --implicit-check-not=warning

// CHECK: ld.lld
// CHECK-SAME: "--thinlto-distributor=dist.exe"
// CHECK-SAME: "--thinlto-remote-compiler={{.*}}clang
// CHECK-SAME: "--thinlto-distributor-arg=distarg1"
// CHECK-SAME: "--thinlto-distributor-arg=distarg2"
// CHECK-SAME: "--thinlto-distributor-arg=distarg3"

// RUN: %clang -flto=thin %s -### -fuse-ld=lld --target=x86_64-linux-gnu \
// RUN: -Xthinlto-distributor=a1 -Xthinlto-distributor=a2,a3 \
// RUN: -fthinlto-distributor=d.exe -Werror 2>&1 | \
// RUN: FileCheck %s --check-prefix=FORWARD

// FORWARD: ld.lld
// FORWARD-SAME: "--thinlto-distributor=d.exe"
// FORWARD-SAME: "--thinlto-remote-compiler={{.*}}clang{{[^\"]*}}"
// FORWARD-SAME: "--thinlto-distributor-arg=a1"
// FORWARD-SAME: "--thinlto-distributor-arg=a2"
// FORWARD-SAME: "--thinlto-distributor-arg=a3"

/// Check that options are not added without --thinlto-distributor= and
/// that there is an unused option warning issued for -Xthinlto-distributor=
/// options. We specify -flto here as these options should be unaffected by it.
// RUN: %clang -### @%t.rsp -flto=thin %s 2>&1 | \
// RUN: FileCheck %s --check-prefixes=NONE,NOMORE --implicit-check-not=warning

// NONE: warning: argument unused during compilation: '-Xthinlto-distributor=distarg1'
// NONE: warning: argument unused during compilation: '-Xthinlto-distributor=distarg2,distarg3'
// NONE: ld.lld
// NOMORE-NOT: distributor
// NOMORE-NOT: remote-compiler
/// that a warning is issued for unused -Xthinlto-distributor options.
// RUN: %clang -flto=thin %s -### -fuse-ld=lld --target=x86_64-linux-gnu \
// RUN: -Xthinlto-distributor=a1 -Xthinlto-distributor=a2,a3 2>&1 | \
// RUN: FileCheck %s --check-prefix=NODIST --implicit-check-not=distributor \
// RUN: --implicit-check-not=remote-compiler

// NODIST: warning: argument unused during compilation: '-Xthinlto-distributor=a1'
// NODIST: warning: argument unused during compilation: '-Xthinlto-distributor=a2,a3'
// NODIST: ld.lld

/// Check the expected arguments are forwarded by default with only
/// --thinlto-distributor=.
// RUN: %clang --target=x86_64-linux-gnu -fthinlto-distributor=dist.exe \
// RUN: -fuse-ld=lld -Werror -### %s 2>&1 | \
// RUN: FileCheck %s --check-prefixes=DEFAULT,NOMORE
// RUN: %clang -flto=thin %s -### -fuse-ld=lld --target=x86_64-linux-gnu \
// RUN: -fthinlto-distributor=d.exe -Werror 2>&1 | \
// RUN: FileCheck %s --check-prefix=DEFAULT --implicit-check-not=distributor \
// RUN: --implicit-check-not=remote-compiler

// DEFAULT: ld.lld
// DEFAULT-SAME: "--thinlto-distributor=dist.exe"
// DEFAULT-SAME: "--thinlto-remote-compiler={{.*}}clang
// DEFAULT-SAME: "--thinlto-distributor=d.exe"
// DEFAULT-SAME: "--thinlto-remote-compiler={{.*}}clang{{[^\"]*}}"

/// Check that nothing is forwarded when the compiler is not in LTO mode, and that
/// appropriate unused option warnings are issued.
// RUN: %clang %s -### -fuse-ld=lld --target=x86_64-linux-gnu \
// RUN: -fthinlto-distributor=d.exe 2>&1 | \
// RUN: FileCheck %s --check-prefix=NOFLTO --implicit-check-not=distributor \
// RUN: --implicit-check-not=remote-compiler

// NOFLTO: warning: argument unused during compilation: '-fthinlto-distributor=d.exe'
// NOFLTO: ld.lld