Skip to content

Commit 73a83a7

Browse files
authored
update export llm doc (#12802)
This PR update model profiling doc in Exporting custom LLMs and Exporting LLMs. Specially: 1. removed export-irrevalnt profiling settings and progress (etdump generation and inspector usage) in export doc 2. reformat the log output for better demonstration 3. replace the link from absolute link to relateive one for version control 4. other small updates
1 parent 8bf58fb commit 73a83a7

File tree

3 files changed

+32
-178
lines changed

3 files changed

+32
-178
lines changed

docs/source/llm/export-custom-llm.md

Lines changed: 3 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -316,17 +316,12 @@ lowered graph():
316316
return [aten_view_copy_default_1]
317317
```
318318

319-
### Performance Analysis
319+
### Further Model Analysis and Debugging
320320

321-
Through the ExecuTorch Developer Tools, users are able to profile model execution, giving timing information for each operator in the model.
322-
323-
#### Prerequisites
324-
325-
##### ETRecord generation (Optional)
321+
Through the [ExecuTorch's Developer Tools](getting-started.md#performance-analysis), users are able to profile model execution, giving timing information for each operator in the model, doing model numeric debugging, etc.
326322

327323
An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../etrecord.rst).
328324

329-
330325
In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_edge_transform_and_lower()` mutates the graph in-place.
331326

332327
```
@@ -346,79 +341,4 @@ generate_etrecord(etrecord_path, edge_manager_copy, et_program)
346341

347342
Run the export script and the ETRecord will be generated as `etrecord.bin`.
348343

349-
##### ETDump generation
350-
351-
An ETDump is an artifact generated at runtime containing a trace of the model execution. For more information, see [the ETDump docs](../etdump.md).
352-
353-
Include the ETDump header and namespace in your code.
354-
```cpp
355-
// main.cpp
356-
357-
#include <executorch/devtools/etdump/etdump_flatcc.h>
358-
359-
using executorch::etdump::ETDumpGen;
360-
using torch::executor::etdump_result;
361-
```
362-
363-
Create an Instance of the ETDumpGen class and pass it to the Module constructor.
364-
```cpp
365-
std::unique_ptr<ETDumpGen> etdump_gen_ = std::make_unique<ETDumpGen>();
366-
Module model("nanogpt.pte", Module::LoadMode::MmapUseMlockIgnoreErrors, std::move(etdump_gen_));
367-
```
368-
369-
After calling `generate()`, save the ETDump to a file. You can capture multiple
370-
model runs in a single trace, if desired.
371-
```cpp
372-
ETDumpGen* etdump_gen = static_cast<ETDumpGen*>(model.event_tracer());
373-
374-
ET_LOG(Info, "ETDump size: %zu blocks", etdump_gen->get_num_blocks());
375-
etdump_result result = etdump_gen->get_etdump_data();
376-
if (result.buf != nullptr && result.size > 0) {
377-
// On a device with a file system, users can just write it to a file.
378-
FILE* f = fopen("etdump.etdp", "w+");
379-
fwrite((uint8_t*)result.buf, 1, result.size, f);
380-
fclose(f);
381-
free(result.buf);
382-
}
383-
```
384-
385-
Additionally, update CMakeLists.txt to build with Developer Tools and enable events to be traced and logged into ETDump:
386-
387-
```
388-
option(EXECUTORCH_ENABLE_EVENT_TRACER "" ON)
389-
option(EXECUTORCH_BUILD_DEVTOOLS "" ON)
390-
391-
# ...
392-
393-
target_link_libraries(
394-
# ... omit existing ones
395-
etdump) # Provides event tracing and logging
396-
397-
target_compile_options(executorch PUBLIC -DET_EVENT_TRACER_ENABLED)
398-
target_compile_options(portable_ops_lib PUBLIC -DET_EVENT_TRACER_ENABLED)
399-
```
400-
Build and run the runner, you will see a file named “etdump.etdp” is generated. (Note that this time we build in release mode to get around a flatccrt build limitation.)
401-
```bash
402-
(rm -rf cmake-out && mkdir cmake-out && cd cmake-out && cmake -DCMAKE_BUILD_TYPE=Release ..)
403-
cmake --build cmake-out -j10
404-
./cmake-out/nanogpt_runner
405-
```
406-
407-
## Performance debugging and profiling
408-
409-
Once you’ve collected debug artifacts ETDump (and optionally an ETRecord), you can use the Inspector API to view performance information.
410-
411-
```python
412-
from executorch.devtools import Inspector
413-
414-
inspector = Inspector(etdump_path="etdump.etdp")
415-
# If you also generated an ETRecord, then pass that in as well: `inspector = Inspector(etdump_path="etdump.etdp", etrecord="etrecord.bin")`
416-
417-
with open("inspector_out.txt", "w") as file:
418-
inspector.print_data_tabular(file)
419-
```
420-
This prints the performance data in a tabular format in “inspector_out.txt”, with each row being a profiling event. Top rows look like this:
421-
![](../_static/img/llm_manual_print_data_tabular.png)
422-
<a href="../_static/img/llm_manual_print_data_tabular.png" target="_blank">View in full size</a>
423-
424-
To learn more about the Inspector and the rich functionality it provides, see the [Inspector API Reference](../model-inspector.rst).
344+
To learn more about ExecuTorch's Developer Tools, see the [Introduction to the ExecuTorch Developer Tools](../devtools-overview.md).

0 commit comments

Comments
 (0)