Skip to content

Commit 47acf81

Browse files
committed
docs: clarify custom event system rationale over emit
1 parent b785750 commit 47acf81

File tree

1 file changed

+26
-42
lines changed

1 file changed

+26
-42
lines changed
Lines changed: 26 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,43 @@
1-
# Custom Event System Over Tracing
1+
# Custom Event System
22

33
## Context
44

5-
http-nu needs structured logging with:
6-
- Rich request/response data (headers, query params, timing)
7-
- Multiple output formats (human-readable terminal, JSONL)
5+
http-nu needs structured logging:
6+
- Request/response lifecycle (headers, timing, bytes)
7+
- Two output formats: human-readable terminal, JSONL for tooling
88
- High throughput without blocking request handling
9-
- Works with `cargo install` without special flags
9+
- Works with `cargo install` (no special flags)
1010

11-
Evaluated options:
11+
## Options
1212

13-
| | **log** | **tracing** | **emit** | **fastrace** | **custom** |
14-
|---|---------|-------------|----------|--------------|------------|
15-
| **Structured data** | ❌ kv unstable since 2019 | ❌ valuable unstable since 2022 | ✅ Native serde |||
16-
| **Your structs/enums** ||||||
17-
| **cargo install works** || ❌ needs RUSTFLAGS ||||
18-
| **API complexity** | Minimal | High | Medium | Low | Minimal |
19-
| **Stability** | Stable | Features stuck 3+ years | Stable | API unstable | N/A |
13+
**tracing**: Rich ecosystem, but `valuable` feature (needed for custom structs) requires `RUSTFLAGS="--cfg tracing_unstable"`. Breaks `cargo install`. Rejected.
2014

21-
tracing's `valuable` feature requires `RUSTFLAGS="--cfg tracing_unstable"` for all downstream users. This broke `cargo install http-nu`.
15+
**emit**: Viable. Native serde, stable, works with cargo install. Provides emit_term (human) and emit_file (rolling JSONL to files). However:
16+
- emit_file targets files, not stdout
17+
- We'd need a custom emitter for stdout JSONL anyway
18+
- Rate-limiting for human output not built-in
19+
- Adds dependency for ~300 lines of purpose-built code
20+
21+
**custom**: Typed Event enum, broadcast channel, dedicated handler threads. Simple, fits exact requirements.
2222

2323
## Decision
2424

25-
Custom event system with broadcast channel and dedicated handler threads:
25+
Custom event system. Typed events, broadcast to handler threads:
2626

2727
```rust
2828
pub enum Event {
29-
Request { request_id: Scru128Id, method: String, path: String, ... },
29+
Request { request_id: Scru128Id, request: Box<RequestData> },
3030
Response { request_id: Scru128Id, status: u16, latency_ms: u64, ... },
3131
Complete { request_id: Scru128Id, bytes: u64, duration_ms: u64 },
3232
Started { address: String, startup_ms: u64 },
3333
// ...
3434
}
3535

36-
// Non-blocking emit via broadcast channel
3736
fn emit(event: Event) {
3837
if let Some(tx) = SENDER.get() {
39-
let _ = tx.send(event);
38+
let _ = tx.send(event); // non-blocking
4039
}
4140
}
42-
43-
// Handlers run in dedicated threads
44-
pub fn run_jsonl_handler(rx: broadcast::Receiver<Event>) {
45-
std::thread::spawn(move || { /* blocking_recv + serialize + write */ });
46-
}
4741
```
4842

4943
## Architecture
@@ -57,22 +51,12 @@ Request Path Handler Thread
5751
continue serialize + write
5852
```
5953

60-
### JSONL Handler
61-
- Dedicated thread with `blocking_recv()`
62-
- BufWriter with idle flush (flush when channel empty)
63-
- Sustains 24K+ requests/sec without drops
64-
65-
### Human Handler
66-
- Dedicated thread with `blocking_recv()`
67-
- Rate limited to ~10 requests/sec (human-readable pace)
68-
- Skipped requests tracked, periodically prints `... skipped N requests`
69-
- Once a request is shown, its full lifecycle (Request→Response→Complete) completes
70-
71-
## Rationale
72-
73-
- **Non-blocking emit**: Request path just sends to channel, never waits
74-
- **Dedicated threads**: Serialization and I/O off the async runtime
75-
- **Idle flush**: Responsive under low load, efficient under high load
76-
- **Rate limiting for human**: No point showing 60K req/sec to humans
77-
- **Complete lifecycle**: Skipping happens at Request level; shown requests always complete
78-
- **No unstable features**: Works with plain `cargo install`
54+
**JSONL handler**: Dedicated thread, BufWriter, flushes when channel empty. 24K+ req/sec sustained.
55+
56+
**Human handler**: Dedicated thread, rate-limited to ~10 req/sec. Tracks skipped requests. Once shown, a request's full lifecycle completes.
57+
58+
## Tradeoffs
59+
60+
- Events are cloned at emit (headers→HashMap, IPs→String). Acceptable at current throughput.
61+
- No file rotation, OTLP, or other emit features. Not needed—stdout only.
62+
- ~300 lines to maintain vs external dependency.

0 commit comments

Comments
 (0)