[GR-2092] Add perf-based profiling support for runtime-compiled methods.

PullRequest: graal/21851

Author: dominikmascherbauer

docs/reference-manual/native-image/PerfProfiling.md

@@ -69,11 +69,61 @@ The following command assumes that `native-image` is on the system path and avai
 If it is not installed, refer to the [Getting Started](README.md).
 
 ```bash
-native-image -g <entry_class>
+native-image -g -H:+PreserveFramePointer <entry_class>
 ```
 
 The `-g` option instructs Native Image to produce debug information for the generated binary.
 `perf` can use this debug information, for example, to provide proper names for types and methods in traces.
+The `-H:+PreserveFramePointer` option instructs Native Image to save frame pointers on the stack.
+This allows `perf` to reliably unwind stack frames and reconstruct the call hierarchy.
+
+### Profiling of Runtime-Compiled Methods
+
+Native Image can generate detailed runtime compilation metadata for perf in the [jitdump](https://github.com/torvalds/linux/blob/46a51f4f5edade43ba66b3c151f0e25ec8b69cb6/tools/perf/Documentation/jitdump-specification.txt) format.
+This enables perf profiling of runtime compiled methods, for example for Truffle compilations.
+
+#### jitdump
+
+The jitdump format stores detailed metadata for runtime compiled code. 
+This requires post-processing of the perf data to inject the runtime compilation metadata.
+
+1. Build with jitdump support:
+
+   ```bash
+   native-image -g -H:+PreserveFramePointer -H:+RuntimeDebugInfo -H:RuntimeDebugInfoFormat=jitdump ...
+   ```
+   
+   At image-runtime, the jitdump file _<jitdump_dir>/jit-<pid>.dump_ is created, and runtime compilation metadata is written to it.
+   The output directory can be configured with `-R:RuntimeJitdumpDir=<jitdump_dir>` (defaults to _./jitdump_).
+
+2. Record with perf:
+
+   When recording profiling data, use the `-k 1` option to ensure time-based events are ordered correctly for injection:
+   
+   ```bash
+   perf record -k 1 -o perf.data <your-application>
+   ```
+   
+   If the perf data was not recorded with `-k 1`, injecting runtime compilation metadata from a jitdump file will fail.
+
+3. Inject jitdump into perf data:
+
+   ```bash
+   perf inject -j -i perf.data -o perf.jit.data
+   ```
+   
+   This step:
+    - Locates the jitdump file. 
+    - Generates a _.so_ file for each runtime compilation entry in the jitdump file. 
+    - Injects runtime compilation metadata into the profiling data and stores it in _perf.jit.data_.
+
+4. Inspect profiling data:
+
+   ```bash
+   perf report -i perf.jit.data
+   ```
+   
+   Symbols from the jitdump file appear as coming from _jitted-<pid>-<code_id>.so_, where `code_id` is the index of a compilation entry in the jitdump file.
 
 ## Basic Operations
 
@@ -141,6 +191,79 @@ The `-g` option instructs Native Image to produce debug information for the gene
 
    This command generates a script that can be used for analyzing the recorded trace data.
 
+## Generating Flame Graphs from Profiling Data
+
+[FlameGraph](https://github.com/brendangregg/FlameGraph) is a tool written in Perl that can be used to produce flame graphs from perf profiling data.
+Flame graphs generated by this tool visualize stack samples as interactive SVGs, making it easy to identify hot code paths in an application.
+
+1. Download the tool and record profiling data as described in [Basic Operations](#basic-operations).
+
+   Make sure the profiling data was recorded with `-g` to capture call graphs, otherwise the flame graph will be flat.
+
+2. Fold stacks:
+   ```bash
+   perf script -i perf.data | ./stackcollapse-perf.pl > perf.data.folded
+   ```
+
+3. Render an SVG:
+   ```bash
+   ./flamegraph.pl perf.data.folded > perf.data.svg
+   ```
+   
+4. Open the flame graph:
+
+   Use an application to view the generated SVG file (for example, `firefox`, `chromium`).
+   ```bash
+   firefox perf.data.svg
+   ```
+
+### Highlighting Runtime-Compiled Methods
+
+If the native image supports [profiling of runtime-compiled methods](#profiling-of-runtime-compiled-methods), it is possible to highlight runtime-compiled symbols in the flame graph.
+
+1. Build the native image with jitdump support, record profiling data and inject the jitdump information as described in [jitdump](#jitdump).
+
+2. Fold stacks:
+
+   This involves folding the stacks for the non-jitdump-injected _perf.data_ and the jitdump-injected _perf.jit.data_.
+   ```bash
+   perf script -i perf.data | ./stackcollapse-perf.pl > perf.data.folded
+   perf script -i perf.jit.data | ./stackcollapse-perf.pl > perf.jit.data.folded
+   ```
+
+3. Generate a consistent color palette map:
+   
+   Use the non-jitdump-injected _perf.data.folded_ to create a consistent palette map in _palette.map_ for events in _perf.data_.
+   The first call with `--cp` will create the map while subsequent calls with `--cp` reuse the map for consistent coloring of known events.
+   This also produces a flame graph for the non-jitdump-injected data.
+   ```bash
+   ./flamegraph.pl --cp perf.data.folded > perf.data.svg
+   ```
+
+4. Reuse the color palette map:
+
+   Use the consistent palette for already known events with `--cp` for the jitdump-injected _perf.jit.data.folded_.
+   This is, events already seen in the non-jitdump-injected _perf.data.folded_ get a fixed coloring.
+   New events get a random coloring from the palette selected with the `--color` option (e.g. `mem`).
+   ```bash
+   ./flamegraph.pl --cp --color mem perf.jit.data.folded > perf.jit.data.svg
+   ```
+
+5. Open the flame graph:
+   ```bash
+   firefox perf.jit.data.svg
+   ```
+
+### Generate an Invocation-Time-Ordered Flame Graph
+
+Generate a stack-reversed flame graph with the topmost frames shown at the bottom of the flame graph in order of invocation time.
+Calls appear left-to-right in chronological order, with stack frames in each call arranged top-to-bottom from oldest to newest.
+Events from all threads contributing to the profiling data are shown interleaved.
+```bash
+./flamegraph.pl --reverse perf.data.folded > perf.data.svg
+firefox perf.data.svg
+```
+
 ### Related Documentation
 
 * [Debug Information](DebugInfo.md)

substratevm/CHANGELOG.md

@@ -6,6 +6,7 @@ This changelog summarizes major changes to GraalVM Native Image.
 * (GR-44384) Add size warnings for bundles when individual or cumulative file sizes exceed limits. Configure with options `size-warning-file-limit` and `size-warning-total-limit` to `bundle-create`, sizes in MiB.
 * (GR-43070) Add a new API flag `-Werror` to treat warnings as errors.
 * (GR-69280) Allow use of the `graal.` prefix for options without issuing a warning.
+* (GR-2092) Add jitdump support for recording run-time compilation metadata for perf (see PerfProfiling.md). Can be enabled with `-g -H:+RuntimeDebugInfo -H:RuntimeDebugInfoFormat=jitdump`.
 
 ## GraalVM 25
 * (GR-52276) (GR-61959) Add support for Arena.ofShared().

substratevm/debug/include/gdb_jit_compilation_interface.h

@@ -27,7 +27,9 @@
 #define SVM_NATIVE_GDBJITCOMPILATIONINTERFACE_H
 
 // This header specifies the types used by the GDB JIT compilation interface (see https://sourceware.org/gdb/current/onlinedocs/gdb.html/Declarations.html#Declarations)
-// The implementation of the JIT compilation interface is located in com.oracle.svm.core.debug.GdbJitInterface.
+// The implementation of the JIT compilation interface is located in com.oracle.svm.core.debug.gdb.GdbJitInterface.
+
+#ifdef __linux__
 
 #include <stdint.h>
 
@@ -56,4 +58,6 @@ struct jit_descriptor
   struct jit_code_entry *first_entry;
 };
 
+#endif /* __linux__ */
+
 #endif

substratevm/debug/include/jitdump_entry.h

@@ -0,0 +1,91 @@
+/*
+ * Copyright (c) 2025, 2025, Oracle and/or its affiliates. All rights reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.  Oracle designates this
+ * particular file as subject to the "Classpath" exception as provided
+ * by Oracle in the LICENSE file that accompanied this code.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
+ * or visit www.oracle.com if you need additional information or have any
+ * questions.
+ */
+
+#ifndef SVM_NATIVE_JITDUMPRECORDS_H
+#define SVM_NATIVE_JITDUMPRECORDS_H
+
+#ifdef __linux__
+
+// This header specifies the jitdump entries described in the Jitdump specification (see https://github.com/torvalds/linux/blob/46a51f4f5edade43ba66b3c151f0e25ec8b69cb6/tools/perf/Documentation/jitdump-specification.txt)
+// The implementation of the Jitdump provider generating these entries is located in com.oracle.svm.core.posix.debug.jitdump.JitdumpProvider.
+
+#include <stdint.h>
+
+typedef enum
+{
+  JIT_CODE_LOAD = 0,
+  JIT_CODE_MOVE = 1,
+  JIT_CODE_DEBUG_INFO = 2,
+  JIT_CODE_CLOSE = 3,
+  JIT_CODE_UNWINDING_INFO = 4
+} record_type;
+
+struct file_header
+{
+  uint32_t magic;
+  uint32_t version;
+  uint32_t total_size;
+  uint32_t elf_mach;
+  uint32_t pad1;
+  uint32_t pid;
+  uint64_t timestamp;
+  uint64_t flags;
+};
+
+struct record_header
+{
+  uint32_t id;
+  uint32_t total_size;
+  uint64_t timestamp;
+};
+
+struct code_load_record
+{
+  struct record_header header;
+  uint32_t pid;
+  uint32_t tid;
+  uint64_t vma;
+  uint64_t code_addr;
+  uint64_t code_size;
+  uint64_t code_index;
+};
+
+struct debug_entry
+{
+  uint64_t code_addr;
+  uint32_t line;
+  uint32_t discrim;
+};
+
+struct debug_info_record
+{
+  struct record_header header;
+  uint64_t code_addr;
+  uint64_t nr_entry;
+};
+
+#endif /* __linux__ */
+
+#endif

substratevm/mx.substratevm/mx_substratevm.py

@@ -793,9 +793,10 @@ def jvm_unittest(args):
     return mx_unittest.unittest(['--suite', 'substratevm'] + args)
 
 
-def js_image_test(jslib, bench_location, name, warmup_iterations, iterations, timeout=None, bin_args=None):
+def js_image_test(jslib, bench_location, name, warmup_iterations, iterations, timeout=None, bin_args=None, pre_args=None):
     bin_args = bin_args if bin_args is not None else []
-    jsruncmd = [get_js_launcher(jslib)] + bin_args + [join(bench_location, 'harness.js'), '--', join(bench_location, name + '.js'),
+    pre_args = pre_args if pre_args is not None else []
+    jsruncmd = pre_args + [get_js_launcher(jslib)] + bin_args + [join(bench_location, 'harness.js'), '--', join(bench_location, name + '.js'),
                                       '--', '--warmup-time=' + str(15_000),
                                       '--warmup-iterations=' + str(warmup_iterations),
                                       '--iterations=' + str(iterations)]
@@ -833,10 +834,10 @@ def build_js_lib(native_image):
 def get_js_launcher(jslib):
     return os.path.join(os.path.dirname(jslib), "..", "bin", "js")
 
-def test_js(js, benchmarks, bin_args=None):
+def test_js(js, benchmarks, bin_args=None, pre_args=None):
     bench_location = join(suite.dir, '..', '..', 'js-benchmarks')
     for benchmark_name, warmup_iterations, iterations, timeout in benchmarks:
-        js_image_test(js, bench_location, benchmark_name, warmup_iterations, iterations, timeout, bin_args=bin_args)
+        js_image_test(js, bench_location, benchmark_name, warmup_iterations, iterations, timeout, bin_args=bin_args, pre_args=pre_args)
 
 def test_run(cmds, expected_stdout, timeout=10, env=None):
     stdoutdata = []
@@ -1296,6 +1297,7 @@ def _runtimedebuginfotest(native_image, output_path, with_isolates_only, args=No
         '-H:DebugInfoSourceSearchPath=' + test_source_path,
         '-H:+SourceLevelDebug',
         '-H:+RuntimeDebugInfo',
+        # We rely on '-H:RuntimeDebugInfoFormat' to default to 'objfile', which is required for this test
     ]) + args
 
     mx.log(f"native-image {' '.join(build_args)}")
@@ -1316,6 +1318,7 @@ def run_js_test(eager: bool = False):
                 '-H:+SourceLevelDebug',
                 '-H:+RuntimeDebugInfo',
                 '-H:-LazyDeoptimization' if eager else '-H:+LazyDeoptimization',
+                # We rely on '-H:RuntimeDebugInfoFormat' to default to 'objfile', which is required for this test
             ]) +
             ['-g', '--macro:jsvm-library']
         ))

substratevm/mx.substratevm/suite.py

@@ -1968,6 +1968,7 @@
                 "com.oracle.objectfile.debugentry",
                 "com.oracle.objectfile.debugentry.range",
                 "com.oracle.objectfile.macho",
+                "com.oracle.objectfile.elf",
               ],
 
               "requiresConcealed" : {