|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Build and Test Commands |
| 6 | + |
| 7 | +```bash |
| 8 | +# Build |
| 9 | +make build # Build to bin/osmedeus |
| 10 | +make build-all # Cross-platform builds (linux, darwin, windows) |
| 11 | + |
| 12 | +# Test |
| 13 | +make test-unit # Fast unit tests (no external dependencies) |
| 14 | +make test-integration # Integration tests (requires Docker) |
| 15 | +make test-e2e # E2E CLI tests (requires binary build) |
| 16 | +make test-e2e-ssh # SSH E2E tests (module & step level SSH runner) |
| 17 | +make test-e2e-api # API E2E tests (all endpoints with Redis + seeded DB) |
| 18 | +make test-distributed # Distributed run e2e tests (requires Docker for Redis) |
| 19 | +make test-docker # Docker runner tests |
| 20 | +make test-ssh # SSH runner unit tests (starts test SSH container) |
| 21 | +go test -v ./internal/functions/... # Run tests for specific package |
| 22 | +go test -v -run TestName ./... # Run single test by name |
| 23 | + |
| 24 | +# Development |
| 25 | +make fmt # Format code |
| 26 | +make lint # Run golangci-lint |
| 27 | +make tidy # go mod tidy |
| 28 | +make run # Build and run |
| 29 | + |
| 30 | +# Installation |
| 31 | +make install # Install to $GOBIN (or $GOPATH/bin) |
| 32 | +make swagger # Generate Swagger documentation |
| 33 | + |
| 34 | +# Docker Toolbox |
| 35 | +make docker-toolbox # Build toolbox image (all tools pre-installed) |
| 36 | +make docker-toolbox-run # Start toolbox container |
| 37 | +make docker-toolbox-shell # Enter toolbox container shell |
| 38 | + |
| 39 | +# UI |
| 40 | +make update-ui # Update embedded UI from dashboard build |
| 41 | +``` |
| 42 | + |
| 43 | +## Architecture Overview |
| 44 | + |
| 45 | +Osmedeus is a workflow engine for security automation. It executes YAML-defined workflows with support for multiple execution environments. |
| 46 | + |
| 47 | +### Layered Architecture |
| 48 | + |
| 49 | +``` |
| 50 | +CLI/API (pkg/cli, pkg/server) |
| 51 | + ↓ |
| 52 | +Executor (internal/executor) - coordinates workflow execution |
| 53 | + ↓ |
| 54 | +StepDispatcher - routes to: BashExecutor, FunctionExecutor, ForeachExecutor, ParallelExecutor, RemoteBashExecutor, HTTPExecutor, LLMExecutor |
| 55 | + ↓ |
| 56 | +Runner (internal/runner) - executes commands via: HostRunner, DockerRunner, SSHRunner |
| 57 | +``` |
| 58 | + |
| 59 | +### Core Packages |
| 60 | + |
| 61 | +| Package | Purpose | |
| 62 | +|---------|---------| |
| 63 | +| `internal/core` | Type definitions: Workflow, Step, Trigger, RunnerConfig, ExecutionContext | |
| 64 | +| `internal/parser` | YAML parsing, validation, and caching (Loader) | |
| 65 | +| `internal/executor` | Workflow execution engine with step dispatching | |
| 66 | +| `internal/runner` | Execution environments implementing Runner interface | |
| 67 | +| `internal/template` | `{{Variable}}` interpolation engine | |
| 68 | +| `internal/functions` | Utility functions via Otto JavaScript VM | |
| 69 | +| `internal/scheduler` | Cron, event, and file-watch triggers (fsnotify-based) | |
| 70 | +| `internal/database` | SQLite/PostgreSQL via Bun ORM | |
| 71 | +| `pkg/cli` | Cobra CLI commands | |
| 72 | +| `pkg/server` | Fiber REST API | |
| 73 | +| `internal/snapshot` | Workspace export/import as compressed ZIP archives | |
| 74 | +| `internal/installer` | Binary installation (direct-fetch and Nix modes) | |
| 75 | +| `internal/state` | Run state export for debugging and sharing | |
| 76 | +| `internal/updater` | Self-update functionality via GitHub releases | |
| 77 | + |
| 78 | +### Key Types |
| 79 | + |
| 80 | +```go |
| 81 | +WorkflowKind: "module" | "flow" // module = single unit, flow = orchestrates modules |
| 82 | +StepType: "bash" | "function" | "parallel-steps" | "foreach" | "remote-bash" | "http" | "llm" |
| 83 | +RunnerType: "host" | "docker" | "ssh" |
| 84 | +TriggerType: "cron" | "event" | "watch" | "manual" |
| 85 | +``` |
| 86 | + |
| 87 | +### Decision Routing |
| 88 | + |
| 89 | +Steps support conditional branching via `decision` field with switch/case syntax: |
| 90 | +```yaml |
| 91 | +decision: |
| 92 | + switch: "{{variable}}" |
| 93 | + cases: |
| 94 | + "value1": { goto: step-a } |
| 95 | + "value2": { goto: step-b } |
| 96 | + default: { goto: fallback } |
| 97 | +``` |
| 98 | +Use `goto: _end` to terminate workflow. |
| 99 | + |
| 100 | +### Workflow Execution Flow |
| 101 | + |
| 102 | +1. CLI parses args ▷ loads config from `~/osmedeus-base/osm-settings.yaml` |
| 103 | +2. Parser loads YAML workflow, validates, caches in Loader |
| 104 | +3. Executor initializes context with built-in variables (`{{Target}}`, `{{Output}}`, etc.) |
| 105 | +4. StepDispatcher routes each step to appropriate executor |
| 106 | +5. Runner executes commands, captures output |
| 107 | +6. Exports propagate to subsequent steps |
| 108 | + |
| 109 | +### Template System |
| 110 | + |
| 111 | +- `{{Variable}}` - standard template variables (Target, Output, threads, etc.) |
| 112 | +- `[[variable]]` - foreach loop variables (to avoid conflicts) |
| 113 | +- Functions evaluated via Otto JS runtime: `fileExists()`, `fileLength()`, `trim()`, etc. |
| 114 | + |
| 115 | +## CLI Commands |
| 116 | + |
| 117 | +```bash |
| 118 | +osmedeus run -f <flow> -t <target> # Run flow workflow |
| 119 | +osmedeus run -m <module> -t <target> # Run module workflow |
| 120 | +osmedeus run -m <m1> -m <m2> -t <target> # Run multiple modules in sequence |
| 121 | +osmedeus run -m <module> -t <target> --timeout 2h # With timeout |
| 122 | +osmedeus run -m <module> -t <target> --repeat # Repeat continuously |
| 123 | +osmedeus run -m <module> -T targets.txt -c 5 # Concurrent target scanning |
| 124 | +osmedeus run -m <module> -t <target> -P params.yaml # With params file |
| 125 | +osmedeus workflow list # List available workflows |
| 126 | +osmedeus workflow show <name> # Show workflow details |
| 127 | +osmedeus workflow validate <name> # Validate workflow YAML |
| 128 | +osmedeus func list # List utility functions |
| 129 | +osmedeus func e 'log_info("{{target}}")' # Evaluate function |
| 130 | +osmedeus --usage-example # Show all usage examples |
| 131 | +osmedeus server # Start REST API (see docs/api/ for endpoints) |
| 132 | +osmedeus server --master # Start as distributed master |
| 133 | +osmedeus worker join # Join as distributed worker |
| 134 | +osmedeus install binary --name <name> # Install specific binary |
| 135 | +osmedeus install binary --all # Install all binaries |
| 136 | +osmedeus install binary --name <name> --check # Check if binary is installed |
| 137 | +osmedeus install binary --all --check # Check all binaries status |
| 138 | +osmedeus install binary --nix-build-install # Install binaries via Nix |
| 139 | +osmedeus install binary --nix-installation # Install Nix package manager |
| 140 | +osmedeus install binary --list-registry-nix-build # List Nix binaries |
| 141 | +osmedeus install binary --list-registry-direct-fetch # List direct-fetch binaries |
| 142 | +osmedeus install env # Add binaries to PATH (auto-detects shell) |
| 143 | +osmedeus install env --all # Add to all shell configs |
| 144 | +osmedeus update # Self-update to latest version |
| 145 | +osmedeus update --check # Check for updates without installing |
| 146 | +osmedeus snapshot export <workspace> # Export workspace as ZIP |
| 147 | +osmedeus snapshot import <source> # Import from file or URL |
| 148 | +osmedeus snapshot list # List available snapshots |
| 149 | +osmedeus run -m <module> -t <target> -G # Run with progress bar (shorthand) |
| 150 | +``` |
| 151 | + |
| 152 | +## API Documentation |
| 153 | + |
| 154 | +REST API documentation with curl examples is in `docs/api/`. Key endpoint categories: |
| 155 | +- **Runs**: Create, list, cancel, get steps/artifacts |
| 156 | +- **Workflows**: List, get details, refresh index |
| 157 | +- **Schedules**: Full CRUD + enable/disable/trigger |
| 158 | +- **Assets/Workspaces**: Query discovered data |
| 159 | +- **Event Logs**: Query execution events |
| 160 | +- **Functions**: Execute utility functions via API |
| 161 | +- **Snapshots**: Export/import workspace archives |
| 162 | +- **LLM**: OpenAI-compatible chat completions and embeddings |
| 163 | +- **Install**: Binary registry and installation management |
| 164 | + |
| 165 | +## Adding New Features |
| 166 | + |
| 167 | +**New Step Type**: Add constant in `core/types.go`, create executor implementing `StepExecutor` interface in `internal/executor/`, register in `PluginRegistry` via `dispatcher.go` |
| 168 | + |
| 169 | +**New Runner**: Implement Runner interface in `internal/runner/`, add type constant, register in runner factory |
| 170 | + |
| 171 | +**New CLI Command**: Create in `pkg/cli/`, add to `rootCmd` in `init()` |
| 172 | + |
| 173 | +**New API Endpoint**: Add handler in `pkg/server/handlers/`, register route in `server.go`, document in `docs/api/` |
| 174 | + |
| 175 | +**New Utility Function**: Add Go implementation in `internal/functions/`, register in `otto_runtime.go` |
| 176 | + |
| 177 | +## Architecture Notes |
| 178 | + |
| 179 | +- **Executor**: Fresh instances created per target/request - no global singleton |
| 180 | +- **Step Dispatcher**: Uses plugin registry pattern for extensible step type handling |
| 181 | +- **Scheduler**: File watching uses fsnotify for instant inotify-based notifications |
| 182 | +- **Decision Routing**: Uses switch/case syntax for conditional workflow branching |
0 commit comments