Commit 338ff6a
Create Sqlite Exporter (meta-pytorch#1930)
Summary:
Stack context:
```
Our tracing subscriber has 3 layers:
- File logging
- Scuba
- Sqlite (usually off)
Although the actual Scuba logging is done in a background thread and we are using a non-blocking file writer, we still have a good chunk of work that happens for events & spans. The solution to this, is to create a `UnifiedLayer` that just sends everything into a background worker, that then delivers all traces to each `Exporter` to handle.
In this diff, we will create an initial `UnifiedLayer` and incrementally move each existing layer into an `Exporter`.
To test correctness, we will run both the old and unified implementations for initializing telemetry on a variety of workloads, and ensure that both are producing the same results
```
In this diff, we will implement an `Exporter` meant to replace `get_reloadable_sqlite_layer()`. We will update our correctness test to query the tables at the end of each workload and ensure all rows are the same
Differential Revision: D873637741 parent 7bf3f51 commit 338ff6a
File tree
6 files changed
+700
-34
lines changed- hyperactor_telemetry
- benches
- src
- exporters
6 files changed
+700
-34
lines changed
0 commit comments