Welcome! This repo is the open forum for brainstorming, resource-sharing, and question-asking in the run-up to Experiment 2025, the half-day unconference where AI engineers trade unfiltered lessons from the trenches.
When & Where
🗓 October 30, 2025 ⏰ 09:00 AM– 1:00 PM PT
📍 Frontier Tower, San Francisco
- Crowd-source breakout ideas before we meet in person.
- Share links, demos, or postmortems No slides required.
- Vote on the threads you want to see become live sessions.
- Click the Discussions tab → “New discussion.”
- Tag your post with relevant category label (see below).
- Link docs, blog posts, or gists—anything the community can riff on.
- 👍 Up-vote or interact with ideas you’d discuss IRL.
| Category Labels | What AI-dev topics belong here? |
|---|---|
| 🔍 Knowledge Acquisition / Discovery | Exploratory analysis, AI-powered search, auto data quality checks |
| 💡 Hypothesis Generation | Prompt engineering as hypothesis writing, embedding visualisation to spot ideas, ML-assisted hypothesis tools, “model = hypothesis” debates |
| 🧪 Experiment Design & Justification | A/B or multi-armed bandits, evaluation harnesses, power analysis for token budgets, experiment-tracking stacks, qualification processes |
| 🔄 Feedback Gathering & Integration | Evaluation suites, live-user feedback loops, observability dashboards, RLHF/RLAIF loops, user-telemetry drift alerts |
| 🔬 Post Mortems & Learning | Incident analysis trees, AI-generated retrospectives, result socialization |
| 📈 Scaling | Compute orchestration, cost controls, high volume experiment tracking |