Scenarios Manager/Marketplace

[aleksandarskrbic]

Motivation

Khaos scenarios are plain YAML — no code, no client library — which makes them inherently
shareable. Today the only distribution channels are the bundled scenarios shipped in the binary
and copy-pasting files around. There's no way to discover scenarios other people have written,
publish your own, or build on someone else's work.

The proposal: a simple hosted registry (marketplace) where users can publish scenarios,
discover and fetch scenarios written by others, and collaborate on them — plus first-class
CLI support so fetching a community scenario is as easy as running a bundled one:

khaos search consumer-lag
khaos pull alice/flink-backpressure
khaos run alice/flink-backpressure        # fetch-and-run in one step
khaos push ./my-scenario.yaml             # publish under your namespace

The npm/Docker Hub/Grafana-dashboards model, but for Kafka failure scenarios.

Why this fits Khaos

• Scenarios are declarative YAML validated by khaos validate — the same validation can gate
  publishing, so everything in the registry is guaranteed loadable.
• The interesting knowledge in chaos testing is the scenario design (what failure, what
  timing, what workload shape reproduces a real incident). That's exactly what a registry lets
  people share.
• Real incidents make great scenarios: "here's the YAML that reproduces the rebalance storm we
  hit in prod" is far more useful published than in a gist.

Proposed shape

Registry server

A small hosted service (getkhaos.dev already exists and auto-deploys) exposing:

• Publish: authenticated push of a scenario file under a user namespace
  (<user>/<scenario-name>), server-side validated with the same logic as khaos validate
  before acceptance.
• Fetch: unauthenticated GET of a scenario by user/name (optionally user/name@version).
• Search/browse: by name, description, tags (e.g. consumer-lag, broker-failure,
  kafka-streams), and possibly by which Khaos features a scenario uses.
• Metadata: description, author, version history, download count, Khaos version
  compatibility.

CLI integration

• khaos search <query> — search the registry
• khaos pull <user/name> — download into a local scenarios directory
• khaos run <user/name> — resolve bundled → local → registry, in that order
• khaos push <file> — validate locally, then publish
• khaos login — authenticate (GitHub OAuth is the obvious choice; no separate account system)

Collaboration

Start minimal and lean on existing tools rather than rebuilding them:

• v1: publish, fetch, search, versioning. "Collaboration" = pulling someone's scenario,
  modifying it, publishing under your own namespace with an attribution/forked-from field.
• Later, if demand exists: comments/ratings, org namespaces, scenario collections
  ("Strimzi incident pack"), web UI for browsing on getkhaos.dev.

An alternative worth serious consideration for v1: a Git-backed registry (a
khaos-scenarios GitHub repo where publishing = PR, and the server is just an index/API over
it). Pros: zero auth to build, review-by-PR gives moderation for free, versioning is git.
Cons: higher friction to publish, PR review becomes a bottleneck. Possibly the right first
step before a real push API.

Trust and safety

Scenarios are declarative and don't execute arbitrary code, which keeps risk low, but:

• server-side validation must be strict (schema-valid YAML only, size limits, no surprises)
• scenario configs can contain hostnames/credentials by mistake — push should warn on or strip
  anything that looks like a secret or a non-placeholder bootstrap address
• namespacing + auth prevents impersonation; some moderation/report mechanism eventually needed

Versioning and compatibility

• scenarios declare (or the registry records) the minimum Khaos version / schema version
• the scenario schema evolves; the registry should re-validate or flag scenarios on schema
  changes rather than serving files that newer CLIs reject
• immutable versions (@1.2.0) with a mutable latest pointer, npm-style

Out of scope (for now)

• paid/private scenarios
• arbitrary code or plugin distribution — YAML scenarios only
• a full social platform; discovery + attribution first

Open design questions

1. Dedicated server with a push API, or Git-backed registry (PR-to-publish) as v1?
2. Auth: GitHub OAuth only, or something else?
3. Where does the registry live — under getkhaos.dev, and what's the hosting/storage backend?
4. Namespace model: flat user/name, or orgs/collections from the start?
5. How is Khaos-version compatibility declared and enforced?
6. Should khaos run <user/name> fetch implicitly, or require an explicit pull first
   (supply-chain-style caution vs convenience)?
7. What does the bundled-scenario story become — do bundled scenarios move into the registry
   as the khaos/ namespace?