Skip to content

Commit af6ab38

Browse files
committed
Added perfetto support (CPU + Vulkan) backends
This uses perfetto in process profiling, and will produce a perfetto binary by the end of the inference. This is very useful to help visualise how the handles the inference. Build: cmake -S . -B build-vk -DCMAKE_BUILD_TYPE=RelWithDebInfo -DGGML_VULKAN=ON cmake --build build-vk -j8 Run: GGML_VK_PERF_SILENT=1 GGML_VK_PERF_LOGGER=1 LLAMA_PERFETTO_TRACE=./out.pftrace build-vk/bin/llama-cli -m model.gguf Test: Tested on M4 Mac In detail this patch does the following: 1. Including the `LlamaPerfetto.h` header file, which contains the definitions for the Perfetto-related functions and variables used in this example. 2. Calling the `llama_perfetto_start()` function to start tracing at the beginning of the conversation. 3. Calling the `llama_perfetto_stop_flush()` function to stop tracing and flush the trace after each generation. 4. Adding a call to the `llama_perfetto_trace_begin_with_text()` function to begin an event in Perfetto with a text description of the current evaluation. 5. Adding a call to the `llama_perfetto_trace_end()` function to end the event after each evaluation. 6. Adding a call to the `llama_perfetto_counter_tokens_per_s()` function to update the Perfetto counter for tokens per second during idle periods. 7. Calling the `llama_perfetto_emit_gpu_timeline()` function to emit GPU timeline slices into Perfetto. 8. Adding a call to the `llama_perfetto_print_gpu_stats()` function to print GPU statistics at idle periods. 9. Calling the `llama_perfetto_flush_dump_stats()` function to flush and dump the Perfetto trace stats to a file at idle periods.
1 parent 3d16b29 commit af6ab38

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+243357
-4464
lines changed

.github/ISSUE_TEMPLATE/010-bug-compilation.yml

Lines changed: 0 additions & 87 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/011-bug-results.yml

Lines changed: 0 additions & 101 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/019-bug-misc.yml

Lines changed: 0 additions & 91 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/020-enhancement.yml

Lines changed: 0 additions & 51 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/030-research.yml

Lines changed: 0 additions & 52 deletions
This file was deleted.

0 commit comments

Comments
 (0)