Skip to content

Commit 233a430

Browse files
noahgiftclaude
andcommitted
feat: Add experiment tracking schema for entrenar integration (Refs phase5-experiment-schema)
Implements the experiment schema for ML experiment tracking, designed to integrate with entrenar's ExperimentStorage (ENT-EPIC-001). Components: - ExperimentRecord: experiment_id, name, created_at, config (JSON) - RunRecord: run_id, experiment_id, status, started_at, ended_at, renacer_span_id - MetricRecord: run_id, key, step, value, timestamp (time-series optimized) - ArtifactRecord: run_id, key, cas_hash, size_bytes (CAS storage) - ExperimentStore: in-memory storage with get_metrics_for_run query Quality: - 31 new tests following EXTREME TDD (RED-GREEN-REFACTOR) - Zero clippy warnings - Full serde serialization support 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent d11b1c2 commit 233a430

File tree

14 files changed

+1708
-0
lines changed

14 files changed

+1708
-0
lines changed

Cargo.lock

Lines changed: 4 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,13 @@ resolver = "2"
1515
# Core compute (SIMD - always included)
1616
trueno = "0.7.1" # SIMD fallback (AVX-512/AVX2/SSE2)
1717

18+
# Serialization (experiment tracking schema)
19+
serde = { version = "1.0", features = ["derive"] }
20+
serde_json = "1.0"
21+
22+
# Timestamps for experiment tracking
23+
chrono = { version = "0.4", features = ["serde"] }
24+
1825
# GPU compute (optional - feature-gated to avoid +3.8MB binary bloat)
1926
wgpu = { version = "22", optional = true, features = ["wgsl"] } # GPU compute (Vulkan/Metal/DX12/WebGPU)
2027
bytemuck = { version = "1", features = ["derive"], optional = true } # Safe byte casting for GPU buffers

book/src/SUMMARY.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@
4646
- [SIMD Primitives](./components/simd/simd-primitives.md)
4747
- [CPU Optimization](./components/simd/cpu-optimization.md)
4848

49+
## Experiment Tracking
50+
51+
- [Experiment Schema](./components/experiment/schema.md)
52+
4953
# EXTREME TDD Methodology
5054

5155
- [RED-GREEN-REFACTOR Cycle](./tdd/red-green-refactor.md)
Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Experiment Schema (Phase 5)
2+
3+
The experiment schema provides data structures for ML experiment tracking, designed to integrate with [entrenar](https://github.com/paiml/entrenar)'s `ExperimentStorage`.
4+
5+
## Schema Overview
6+
7+
```
8+
ExperimentRecord (1) ──< RunRecord (N)
9+
10+
├──< MetricRecord (N) [time-series]
11+
└──< ArtifactRecord (N) [CAS]
12+
```
13+
14+
## Record Types
15+
16+
### ExperimentRecord
17+
18+
Root entity representing a tracked experiment.
19+
20+
```rust
21+
use trueno_db::experiment::ExperimentRecord;
22+
23+
// Simple creation
24+
let experiment = ExperimentRecord::new("exp-001", "ResNet Training");
25+
26+
// With configuration
27+
let config = serde_json::json!({
28+
"learning_rate": 0.01,
29+
"batch_size": 32,
30+
"model": "resnet50"
31+
});
32+
33+
let experiment = ExperimentRecord::builder("exp-002", "Custom Config")
34+
.config(config)
35+
.build();
36+
37+
// Access fields
38+
println!("ID: {}", experiment.experiment_id());
39+
println!("Name: {}", experiment.name());
40+
println!("Created: {}", experiment.created_at());
41+
```
42+
43+
### RunRecord
44+
45+
Represents a single execution of an experiment with lifecycle tracking.
46+
47+
```rust
48+
use trueno_db::experiment::{RunRecord, RunStatus};
49+
50+
// Create a run
51+
let mut run = RunRecord::new("run-001", "exp-001");
52+
53+
// Start execution
54+
run.start();
55+
assert_eq!(run.status(), RunStatus::Running);
56+
57+
// Complete with status
58+
run.complete(RunStatus::Success);
59+
assert!(run.ended_at().is_some());
60+
61+
// With renacer span for distributed tracing
62+
let run = RunRecord::builder("run-002", "exp-001")
63+
.renacer_span_id("span-abc-123")
64+
.build();
65+
```
66+
67+
**RunStatus Variants:**
68+
- `Pending` - Created but not started
69+
- `Running` - Currently executing
70+
- `Success` - Completed successfully
71+
- `Failed` - Terminated with error
72+
- `Cancelled` - User/system cancelled
73+
74+
### MetricRecord
75+
76+
Time-series optimized metric storage with step-based ordering.
77+
78+
```rust
79+
use trueno_db::experiment::MetricRecord;
80+
81+
// Log training metrics
82+
for step in 0..100 {
83+
let loss = 1.0 / (step as f64 + 1.0);
84+
let metric = MetricRecord::new("run-001", "loss", step, loss);
85+
// Store metric...
86+
}
87+
88+
// With explicit timestamp
89+
use chrono::{TimeZone, Utc};
90+
let ts = Utc.with_ymd_and_hms(2025, 1, 15, 12, 0, 0).unwrap();
91+
92+
let metric = MetricRecord::builder("run-001", "accuracy", 100, 0.95)
93+
.timestamp(ts)
94+
.build();
95+
```
96+
97+
### ArtifactRecord
98+
99+
Content-addressable storage for model checkpoints and outputs.
100+
101+
```rust
102+
use trueno_db::experiment::ArtifactRecord;
103+
104+
let artifact = ArtifactRecord::new(
105+
"run-001",
106+
"model.pt",
107+
"sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
108+
104_857_600, // 100MB
109+
);
110+
111+
println!("Key: {}", artifact.key());
112+
println!("Hash: {}", artifact.cas_hash());
113+
println!("Size: {} bytes", artifact.size_bytes());
114+
```
115+
116+
## ExperimentStore
117+
118+
In-memory storage with time-series query capabilities.
119+
120+
```rust
121+
use trueno_db::experiment::{
122+
ExperimentStore, ExperimentRecord, RunRecord, MetricRecord, RunStatus,
123+
};
124+
125+
// Create store
126+
let mut store = ExperimentStore::new();
127+
128+
// Add experiment and run
129+
let experiment = ExperimentRecord::new("exp-001", "Training");
130+
store.add_experiment(experiment);
131+
132+
let mut run = RunRecord::new("run-001", "exp-001");
133+
run.start();
134+
store.add_run(run);
135+
136+
// Log metrics during training
137+
for step in 0..100 {
138+
store.add_metric(MetricRecord::new(
139+
"run-001",
140+
"loss",
141+
step,
142+
1.0 / (step as f64 + 1.0),
143+
));
144+
store.add_metric(MetricRecord::new(
145+
"run-001",
146+
"accuracy",
147+
step,
148+
step as f64 / 100.0,
149+
));
150+
}
151+
152+
// Query metrics for visualization (ordered by step)
153+
let loss_curve = store.get_metrics_for_run("run-001", "loss");
154+
assert_eq!(loss_curve.len(), 100);
155+
assert_eq!(loss_curve[0].step(), 0); // First step
156+
assert_eq!(loss_curve[99].step(), 99); // Last step
157+
158+
// Query runs for experiment
159+
let runs = store.get_runs_for_experiment("exp-001");
160+
```
161+
162+
## Serialization
163+
164+
All records support JSON serialization via serde:
165+
166+
```rust
167+
use trueno_db::experiment::MetricRecord;
168+
169+
let metric = MetricRecord::new("run-001", "loss", 50, 0.25);
170+
171+
// Serialize
172+
let json = serde_json::to_string(&metric)?;
173+
174+
// Deserialize
175+
let restored: MetricRecord = serde_json::from_str(&json)?;
176+
assert_eq!(metric.run_id(), restored.run_id());
177+
```
178+
179+
## Integration with entrenar
180+
181+
This schema is designed to be the storage foundation for entrenar's experiment tracking:
182+
183+
```rust
184+
// entrenar writes to trueno-db
185+
let store = ExperimentStore::new();
186+
187+
// ExperimentStorage from entrenar uses this schema
188+
// See: entrenar/docs/specifications/experiment-tracking-spec.md §3.1
189+
```
190+
191+
## Time-Series Optimization
192+
193+
The `get_metrics_for_run` query returns metrics ordered by step, enabling:
194+
195+
- Loss curve visualization
196+
- Learning rate schedules
197+
- Accuracy progression
198+
- Any time-series metric analysis
199+
200+
```rust
201+
// Metrics are always returned in step order
202+
let metrics = store.get_metrics_for_run("run-001", "loss");
203+
for (i, m) in metrics.iter().enumerate() {
204+
assert_eq!(m.step(), i as u64);
205+
}
206+
```

docs/roadmaps/roadmap.yaml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,3 +94,27 @@ roadmap:
9494
- testing
9595
- e2e
9696
notes: null
97+
- id: phase5-experiment-schema
98+
github_issue: 4
99+
item_type: task
100+
title: 'Phase 5: Experiment Tracking Schema for entrenar'
101+
status: completed
102+
priority: high
103+
assigned_to: null
104+
created: 2025-11-30T12:00:00.000000000+00:00
105+
updated: 2025-11-30T12:00:00.000000000+00:00
106+
spec: null
107+
acceptance_criteria:
108+
- experiments table schema with metadata columns
109+
- runs table schema with status and timing
110+
- metrics time-series optimized for range queries
111+
- artifacts table with CAS references
112+
phases: []
113+
subtasks: []
114+
estimated_effort: medium
115+
labels:
116+
- phase-5
117+
- entrenar
118+
- experiment-tracking
119+
- schema
120+
notes: 'Cross-project: ENT-EPIC-001 (entrenar experiment tracking spec v1.8.0)'

0 commit comments

Comments
 (0)