Skip to content

Conversation

@tobias-stadler
Copy link
Contributor

This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like opt-viewer.

The first summary strategy is --inline-callees, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Created using spr 1.3.7-wip

[skip ci]
Created using spr 1.3.7-wip

// Use one cl::opt per Strategy, because future strategies might need to take
// per-strategy parameters.
static cl::opt<bool> EnableInlineSummaryOpt(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about making this a cl::list<std::unique_ptr<SummaryStrategy>>? I think that would nicely support repeated summaries, like an optimization pipeline of sorts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the idea of having remark "optimization pipelines", but I'd prefer to keep this complexity out of the individual commands (we would need custom argument parsing, help printing, etc. which seems like a hassle to maintain). Maybe we can instead build a subcommand llvm-remarkutil pipe where one can pass little scripts e.g. filter; summary;count;<some other future command>, then the pipelines could work for all the other subcommands as well.

tobias-stadler added a commit to tobias-stadler/llvm-project that referenced this pull request Sep 26, 2025
This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like `opt-viewer`.

The first summary strategy is `--inline-callees`, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Pull Request: llvm#160549
Created using spr 1.3.8-wip

[skip ci]
Created using spr 1.3.8-wip
@tobias-stadler tobias-stadler changed the base branch from users/tobias-stadler/spr/main.llvm-remarkutil-introduce-summary-tool to main October 27, 2025 16:54
@tobias-stadler tobias-stadler merged commit 128214f into main Oct 27, 2025
17 of 18 checks passed
@tobias-stadler tobias-stadler deleted the users/tobias-stadler/spr/llvm-remarkutil-introduce-summary-tool branch October 27, 2025 16:54
llvm-sync bot pushed a commit to arm/arm-toolchain that referenced this pull request Oct 27, 2025
This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like `opt-viewer`.

The first summary strategy is `--inline-callees`, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Pull Request: llvm/llvm-project#160549
@llvm-ci
Copy link
Collaborator

llvm-ci commented Oct 27, 2025

LLVM Buildbot has detected a new failure on builder clang-m68k-linux-cross running on suse-gary-m68k-cross while building llvm at step 5 "ninja check 1".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/27/builds/18106

Here is the relevant piece of the build log for the reference
Step 5 (ninja check 1) failure: stage 1 checked (failure)
...
[312/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/DeclRefExpr.cpp.o
[313/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/ImplicitCtorInitializer.cpp.o
[314/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/ImplicitCtor.cpp.o
[315/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/DeductionGuide.cpp.o
[316/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/DependencyScanning/DependencyScannerTest.cpp.o
[317/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/InitListExprPreOrder.cpp.o
[318/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/InitListExprPreOrderNoQueue.cpp.o
[319/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RewriterTest.cpp.o
[320/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/IntegerLiteral.cpp.o
[321/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AST/ASTImporterTest.cpp.o
FAILED: tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AST/ASTImporterTest.cpp.o 
/usr/bin/c++ -DGTEST_HAS_RTTI=0 -DLLVM_BUILD_STATIC -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/unittests -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang/unittests -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/tools/clang/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/stage1/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/llvm/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang/unittests/Tooling -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/third-party/unittest/googletest/include -I/var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/third-party/unittest/googlemock/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-uninitialized -Wno-nonnull -Wno-class-memaccess -Wno-dangling-reference -Wno-redundant-move -Wno-pessimizing-move -Wno-array-bounds -Wno-stringop-overread -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -O3 -DNDEBUG -std=c++17  -Wno-variadic-macros -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -Wno-suggest-override -MD -MT tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AST/ASTImporterTest.cpp.o -MF tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AST/ASTImporterTest.cpp.o.d -o tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/AST/ASTImporterTest.cpp.o -c /var/lib/buildbot/workers/suse-gary-m68k-cross/clang-m68k-linux-cross/llvm/clang/unittests/AST/ASTImporterTest.cpp
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[322/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/ParenExpr.cpp.o
[323/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/LambdaDefaultCapture.cpp.o
[324/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/LambdaTemplateParams.cpp.o
[325/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/MemberPointerTypeLoc.cpp.o
[326/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/NestedNameSpecifiers.cpp.o
[327/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RangeSelectorTest.cpp.o
[328/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/TemplateArgumentLocTraverser.cpp.o
[329/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/LexicallyOrderedRecursiveASTVisitorTest.cpp.o
[330/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTestDeclVisitor.cpp.o
[331/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/ReplacementsYamlTest.cpp.o
[332/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/Attr.cpp.o
[333/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTestTypeLocVisitor.cpp.o
[334/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/TraversalScope.cpp.o
[335/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/LambdaExpr.cpp.o
[336/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/InitListExprPostOrder.cpp.o
[337/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/InitListExprPostOrderNoQueue.cpp.o
[338/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RefactoringActionRulesTest.cpp.o
[339/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RefactoringCallbacksTest.cpp.o
[340/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RefactoringTest.cpp.o
[341/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/ASTMatchers/ASTMatchersNodeTest.cpp.o
[342/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/ToolingTest.cpp.o
[343/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTestPostOrderVisitor.cpp.o
[344/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/SourceCodeBuildersTest.cpp.o
[345/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/CallbacksCallExpr.cpp.o
[346/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/CallbacksBinaryOperator.cpp.o
[347/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/CallbacksCompoundAssignOperator.cpp.o
[348/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/CallbacksLeaf.cpp.o
[349/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/RecursiveASTVisitorTests/CallbacksUnaryOperator.cpp.o
[350/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/ASTMatchers/ASTMatchersNarrowingTest.cpp.o
[351/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/SourceCodeTest.cpp.o
[352/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/StencilTest.cpp.o
[353/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/Tooling/TransformerTest.cpp.o
[354/1210] Building CXX object tools/clang/unittests/CMakeFiles/AllClangUnitTests.dir/ASTMatchers/ASTMatchersTraversalTest.cpp.o
ninja: build stopped: subcommand failed.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Oct 27, 2025

LLVM Buildbot has detected a new failure on builder lldb-remote-linux-ubuntu running on as-builder-9 while building llvm at step 16 "test-check-lldb-api".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/195/builds/16556

Here is the relevant piece of the build log for the reference
Step 16 (test-check-lldb-api) failure: Test just built components: check-lldb-api completed (failure)
...
PASS: lldb-api :: types/TestRecursiveTypes.py (1308 of 1317)
UNSUPPORTED: lldb-api :: windows/launch/missing-dll/TestMissingDll.py (1309 of 1317)
PASS: lldb-api :: types/TestIntegerTypeExpr.py (1310 of 1317)
UNSUPPORTED: lldb-api :: windows/launch/replace-dll/TestReplaceDLL.py (1311 of 1317)
PASS: lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py (1312 of 1317)
PASS: lldb-api :: types/TestShortType.py (1313 of 1317)
PASS: lldb-api :: types/TestLongTypes.py (1314 of 1317)
PASS: lldb-api :: types/TestShortTypeExpr.py (1315 of 1317)
PASS: lldb-api :: types/TestLongTypesExpr.py (1316 of 1317)
TIMEOUT: lldb-api :: python_api/process/cancel_attach/TestCancelAttach.py (1317 of 1317)
******************** TEST 'lldb-api :: python_api/process/cancel_attach/TestCancelAttach.py' FAILED ********************
Script:
--
/usr/bin/python3.12 /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib --env LLVM_INCLUDE_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include --env LLVM_TOOLS_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin --libcxx-include-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include/c++/v1 --libcxx-include-target-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include/aarch64-unknown-linux-gnu/c++/v1 --libcxx-library-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib/aarch64-unknown-linux-gnu --arch aarch64 --build-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex --lldb-module-cache-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin/lldb --compiler /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/bin/clang --dsymutil /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin --lldb-obj-root /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/tools/lldb --lldb-libs-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib --cmake-build-type Release --platform-url connect://jetson-agx-2198.lab.llvm.org:1234 --platform-working-dir /home/ubuntu/lldb-tests --sysroot /mnt/fs/jetson-agx-ubuntu --env ARCH_CFLAGS=-mcpu=cortex-a78 --platform-name remote-linux --skip-category=lldb-server /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/llvm-project/lldb/test/API/python_api/process/cancel_attach -p TestCancelAttach.py
--
Exit Code: -9
Timeout: Reached timeout of 600 seconds

Command Output (stdout):
--
lldb version 22.0.0git (https://github.com/llvm/llvm-project.git revision 128214f3b2a4b470a4b45f9b2eece7a439d795d7)
  clang revision 128214f3b2a4b470a4b45f9b2eece7a439d795d7
  llvm revision 128214f3b2a4b470a4b45f9b2eece7a439d795d7

--
Command Output (stderr):
--
WARNING:root:Custom libc++ is not supported for remote runs: ignoring --libcxx arguments
FAIL: LLDB (/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/bin/clang-aarch64) :: test_scripted_implementation (TestCancelAttach.AttachCancelTestCase.test_scripted_implementation)

--

********************
Slowest Tests:
--------------------------------------------------------------------------
600.04s: lldb-api :: python_api/process/cancel_attach/TestCancelAttach.py
123.37s: lldb-api :: functionalities/progress_reporting/TestProgressReporting.py
70.13s: lldb-api :: commands/process/attach/TestProcessAttach.py
64.87s: lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx-simulators/string/TestDataFormatterLibcxxStringSimulator.py
60.76s: lldb-api :: commands/command/script_alias/TestCommandScriptAlias.py
35.24s: lldb-api :: functionalities/single-thread-step/TestSingleThreadStepTimeout.py
34.27s: lldb-api :: functionalities/completion/TestCompletion.py
26.77s: lldb-api :: python_api/watchpoint/watchlocation/TestTargetWatchAddress.py
24.84s: lldb-api :: commands/statistics/basic/TestStats.py
21.34s: lldb-api :: functionalities/gdb_remote_client/TestGDBRemoteClient.py
20.63s: lldb-api :: functionalities/gdb_remote_client/TestPlatformClient.py
18.93s: lldb-api :: functionalities/thread/state/TestThreadStates.py
18.75s: lldb-api :: commands/dwim-print/TestDWIMPrint.py
14.36s: lldb-api :: commands/expression/expr-in-syscall/TestExpressionInSyscall.py

dvbuka pushed a commit to dvbuka/llvm-project that referenced this pull request Oct 27, 2025
This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like `opt-viewer`.

The first summary strategy is `--inline-callees`, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Pull Request: llvm#160549
@llvm-ci
Copy link
Collaborator

llvm-ci commented Oct 28, 2025

LLVM Buildbot has detected a new failure on builder clang-ppc64le-linux-multistage running on ppc64le-clang-multistage-test while building llvm at step 10 "build stage 2".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/76/builds/13307

Here is the relevant piece of the build log for the reference
Step 10 (build stage 2) failure: 'ninja' (failure) (timed out)
...
[6271/6569] Linking CXX executable bin/clang-sycl-linker
[6272/6569] Linking CXX executable bin/llvm-reduce
[6273/6569] Linking CXX executable bin/llvm-split
[6274/6569] Linking CXX executable bin/llvm-lipo
[6275/6569] Linking CXX executable bin/dsymutil
[6276/6569] Linking CXX executable bin/llvm-libtool-darwin
[6277/6569] Linking CXX executable bin/llvm-gsymutil
[6278/6569] Linking CXX executable bin/opt
[6279/6569] Building CXX object tools/bugpoint-passes/CMakeFiles/BugpointPasses.dir/TestPasses.cpp.o
[6280/6569] Linking CXX shared module lib/BugpointPasses.so
command timed out: 1200 seconds without output running [b'ninja'], attempting to kill
process killed by signal 9
program finished with exit code -1
elapsedTime=2134.493142

Lukacma pushed a commit to Lukacma/llvm-project that referenced this pull request Oct 29, 2025
This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like `opt-viewer`.

The first summary strategy is `--inline-callees`, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Pull Request: llvm#160549
aokblast pushed a commit to aokblast/llvm-project that referenced this pull request Oct 30, 2025
This tool provides a harness for implementing different strategies that
summarize many remarks (possibly from multiple translation units) into
new summary remarks. The remark summaries can then be viewed using tools
like `opt-viewer`.

The first summary strategy is `--inline-callees`, which generates
remarks that summarize the per-callee inline statistics for functions
that appear in inling remarks. This is useful for troubleshooting
inlining issues/regressions on large codebases.

Pull Request: llvm#160549
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants