Skip to content

Commit 2a7e584

Browse files
rusenaskrusenask
authored
Merge pull request #836 from keel-hq/feature/000156-explore-the-repo
explore the repo structure and create ARCHITECTURE.md fil...
2 parents 8100cd1 + 1755563 commit 2a7e584

File tree

1 file changed

+301
-0
lines changed

1 file changed

+301
-0
lines changed

‎ARCHITECTURE.md‎

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
# Keel Architecture Guide
2+
3+
> A comprehensive guide for AI agents and developers to understand and navigate the Keel codebase.
4+
5+
## Quick Start - Read These First
6+
7+
If you're new to this codebase, read these files in order:
8+
9+
1. `types/types.go` - Core domain types (Repository, Event, Policy, TriggerType)
10+
2. `cmd/keel/main.go` - Application entry point, shows how everything connects
11+
3. `provider/provider.go` - Provider interface definition
12+
4. `trigger/poll/watcher.go` - Example trigger implementation
13+
14+
## What is Keel?
15+
16+
Keel is a **Kubernetes deployment automation tool** written in Go. It watches container registries for new image versions and automatically updates Kubernetes deployments based on configured policies.
17+
18+
```
19+
┌─────────────────────────────────────────────────────────────────────────┐
20+
│ TRIGGERS │
21+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────────┐ │
22+
│ │ Poll Trigger │ │ PubSub (GCR) │ │ Webhooks (DockerHub, etc.) │ │
23+
│ └──────┬───────┘ └──────┬───────┘ └──────────────┬───────────────┘ │
24+
└─────────┼─────────────────┼──────────────────────────┼──────────────────┘
25+
│ │ │
26+
└─────────────────┴────────────┬─────────────┘
27+
│
28+
▼
29+
┌─────────────────┐
30+
│ Event │
31+
│ (Repository + │
32+
│ new version) │
33+
└────────┬────────┘
34+
│
35+
┌───────────────────┴───────────────────┐
36+
│ │
37+
▼ ▼
38+
┌─────────────────┐ ┌─────────────────┐
39+
│ Kubernetes │ │ Helm3 │
40+
│ Provider │ │ Provider │
41+
└────────┬────────┘ └────────┬────────┘
42+
│ │
43+
▼ ▼
44+
┌─────────────────┐ ┌─────────────────┐
45+
│ Update Deployment│ │ Update Release │
46+
│ (if policy match)│ │ (if policy match)│
47+
└─────────────────┘ └─────────────────┘
48+
```
49+
50+
## Directory Structure
51+
52+
| Directory | Purpose | Key Files |
53+
|-----------|---------|-----------|
54+
| `cmd/keel/` | **Entry point** - Application startup, wiring | `main.go` |
55+
| `provider/` | **Deployment handlers** - Update K8s/Helm resources | `provider.go`, `kubernetes/`, `helm3/` |
56+
| `trigger/` | **Event sources** - Detect new image versions | `poll/`, `pubsub/` |
57+
| `pkg/http/` | **HTTP server + webhooks** - REST API, registry webhooks | `http.go`, `*_webhook_trigger.go` |
58+
| `types/` | **Domain types** - Core data structures | `types.go` |
59+
| `internal/policy/` | **Version matching** - Semver, glob, force, regexp | `policy.go`, `semver.go` |
60+
| `extension/` | **Plugins** - Notifications, credentials helpers | `notification/`, `credentialshelper/` |
61+
| `approvals/` | **Approval workflow** - Manual approval before updates | `approvals.go` |
62+
| `bot/` | **Chat bots** - Slack/HipChat for approvals | `bot.go`, `slack/`, `hipchat/` |
63+
| `registry/` | **Registry client** - Docker registry API | `registry.go` |
64+
| `secrets/` | **K8s secrets** - Extract registry credentials | `secrets.go` |
65+
| `ui/` | **Web dashboard** - Vue.js frontend | `src/` |
66+
| `pkg/store/` | **Persistence** - SQLite database | `sql/` |
67+
| `pkg/auth/` | **Authentication** - Basic auth, JWT | |
68+
| `internal/k8s/` | **K8s utilities** - Watchers, resource cache | |
69+
| `chart/` | **Helm chart** - Deploy Keel itself | |
70+
| `constants/` | **Environment variables** - Config constants | |
71+
| `version/` | **Build info** - Version, revision | |
72+
| `util/` | **Utilities** - Image parsing, etc. | |
73+
74+
## Core Concepts
75+
76+
### 1. Providers
77+
78+
Providers handle deployment updates for different platforms. They implement the `Provider` interface:
79+
80+
```go
81+
// provider/provider.go
82+
type Provider interface {
83+
Submit(event types.Event) error // Process an update event
84+
TrackedImages() ([]*types.TrackedImage, error) // List monitored images
85+
GetName() string
86+
Stop()
87+
}
88+
```
89+
90+
**Available providers:**
91+
- `provider/kubernetes/` - Native Kubernetes Deployments, StatefulSets, DaemonSets, CronJobs
92+
- `provider/helm3/` - Helm v3 releases (enabled via `HELM3_PROVIDER=true`)
93+
94+
### 2. Triggers
95+
96+
Triggers detect new image versions and emit `Event` objects:
97+
98+
```go
99+
// types/types.go
100+
type Event struct {
101+
Repository Repository // Image info (host, name, tag, digest)
102+
CreatedAt time.Time
103+
TriggerName string // "poll", "pubsub", "webhook", etc.
104+
}
105+
```
106+
107+
**Available triggers:**
108+
- `trigger/poll/` - Periodically polls registries for new tags
109+
- `trigger/pubsub/` - Google Cloud Pub/Sub for GCR events
110+
- `pkg/http/*_webhook_trigger.go` - Webhooks from DockerHub, Azure, GitHub, Harbor, Quay, JFrog
111+
112+
### 3. Policies
113+
114+
Policies determine which version updates are allowed. Set via `keel.sh/policy` annotation:
115+
116+
```go
117+
// internal/policy/policy.go - Policy types
118+
type PolicyType int
119+
const (
120+
PolicyTypeNone PolicyType = iota
121+
PolicyTypeSemver // major, minor, patch, all
122+
PolicyTypeForce // always update (for :latest)
123+
PolicyTypeGlob // glob pattern matching
124+
PolicyTypeRegexp // regex pattern matching
125+
)
126+
```
127+
128+
**Policy examples:**
129+
- `keel.sh/policy: major` - Allow major version bumps (1.x.x → 2.x.x)
130+
- `keel.sh/policy: minor` - Allow minor bumps (1.1.x → 1.2.x)
131+
- `keel.sh/policy: patch` - Allow patch bumps only (1.1.1 → 1.1.2)
132+
- `keel.sh/policy: force` - Always update (for mutable tags like `latest`)
133+
- `keel.sh/policy: glob:release-*` - Match glob patterns
134+
135+
### 4. Notifications
136+
137+
Extensible notification system using sender registration pattern:
138+
139+
```go
140+
// extension/notification/notification.go
141+
func RegisterSender(name string, s Sender) { ... }
142+
```
143+
144+
**Available senders:** Slack, Teams, Discord, Mattermost, HipChat, Mail, Webhook, Auditor
145+
146+
Notifications are registered via blank imports in `cmd/keel/main.go`:
147+
```go
148+
_ "github.com/keel-hq/keel/extension/notification/slack"
149+
```
150+
151+
### 5. Approvals
152+
153+
Manual approval workflow before updates proceed:
154+
155+
```go
156+
// approvals/approvals.go
157+
type Manager interface {
158+
Create(r *types.Approval) error
159+
Approve(id, voter string) (*types.Approval, error)
160+
Reject(id string) (*types.Approval, error)
161+
// ...
162+
}
163+
```
164+
165+
Set via `keel.sh/approvals: "2"` annotation to require N approvals.
166+
167+
## Data Flow
168+
169+
1. **Trigger detects new version** → Creates `types.Event`
170+
2. **Event submitted to Providers** → `provider.Submit(event)`
171+
3. **Provider checks policies** → `internal/policy/` evaluates if update allowed
172+
4. **Approval check** → If approvals required, waits for manual approval
173+
5. **Deployment updated** → Provider patches K8s resource or Helm release
174+
6. **Notifications sent** → Slack/webhook/etc. notified of update
175+
176+
## Key Annotations
177+
178+
| Annotation | Purpose | Example |
179+
|------------|---------|---------|
180+
| `keel.sh/policy` | Update policy | `minor`, `patch`, `force`, `glob:v*` |
181+
| `keel.sh/trigger` | Trigger type | `poll` (default: webhooks) |
182+
| `keel.sh/pollSchedule` | Poll frequency | `@every 5m` |
183+
| `keel.sh/approvals` | Required approvals | `2` |
184+
| `keel.sh/approvalDeadline` | Approval timeout (hours) | `24` |
185+
| `keel.sh/notify` | Override notification channel | `#deployments` |
186+
| `keel.sh/matchTag` | Force tag matching | `true` |
187+
| `keel.sh/matchPreRelease` | Match pre-release versions | `true` |
188+
| `keel.sh/digest` | Track by digest (internal) | SHA256 digest |
189+
| `keel.sh/imagePullSecret` | Registry credentials secret | `my-registry-secret` |
190+
| `keel.sh/releaseNotes` | Release notes URL | `https://...` |
191+
| `keel.sh/initContainers` | Track init containers | `true` |
192+
193+
## Environment Variables
194+
195+
| Variable | Purpose | Default |
196+
|----------|---------|---------|
197+
| `PUBSUB` | Enable GCR Pub/Sub trigger | (disabled) |
198+
| `POLL` | Enable/disable poll trigger | `1` (enabled) |
199+
| `PROJECT_ID` | GCP project for Pub/Sub | |
200+
| `HELM3_PROVIDER` | Enable Helm3 provider | `false` |
201+
| `DEBUG` | Enable debug logging | `false` |
202+
| `NOTIFICATION_LEVEL` | Min notification level | `info` |
203+
| `BASIC_AUTH_USER` | HTTP basic auth username | |
204+
| `BASIC_AUTH_PASSWORD` | HTTP basic auth password | |
205+
| `AUTHENTICATED_WEBHOOKS` | Require auth for webhooks | `false` |
206+
| `DOCKER_REGISTRY_CFG` | Default registry credentials | |
207+
| `XDG_DATA_HOME` | Data directory (SQLite) | `/data` |
208+
| `UI_DIR` | Web UI static files | `www` |
209+
| `KUBERNETES_CONFIG` | Kubeconfig path | `~/.kube/config` |
210+
| `POLL_DEFAULTSCHEDULE` | Default poll interval | `@every 1m` |
211+
212+
## Extension Points
213+
214+
### Adding a New Notification Sender
215+
216+
1. Create `extension/notification/mynotifier/mynotifier.go`
217+
2. Implement `notification.Sender` interface
218+
3. Register via `init()`:
219+
```go
220+
func init() {
221+
notification.RegisterSender("mynotifier", &sender{})
222+
}
223+
```
224+
4. Add blank import in `cmd/keel/main.go`:
225+
```go
226+
_ "github.com/keel-hq/keel/extension/notification/mynotifier"
227+
```
228+
229+
### Adding a New Webhook Trigger
230+
231+
1. Create `pkg/http/myregistry_webhook_trigger.go`
232+
2. Parse the webhook payload, extract repository/tag info
233+
3. Create `types.Event` and call `providers.Submit(event)`
234+
4. Register route in `pkg/http/http.go`
235+
236+
### Adding a New Provider
237+
238+
1. Create `provider/myprovider/`
239+
2. Implement `provider.Provider` interface
240+
3. Initialize in `cmd/keel/main.go` `setupProviders()`
241+
242+
### Adding a New Credentials Helper
243+
244+
1. Create `extension/credentialshelper/myhelper/`
245+
2. Implement `credentialshelper.CredentialsHelper` interface
246+
3. Register via `init()` and blank import in `main.go`
247+
248+
## Building & Running
249+
250+
```bash
251+
# Build
252+
make build
253+
254+
# Run locally (outside cluster)
255+
make run
256+
257+
# Run tests
258+
make test
259+
260+
# Build Docker image
261+
make image
262+
```
263+
264+
## Common Tasks
265+
266+
| Task | Where to Look |
267+
|------|---------------|
268+
| Add new webhook support | `pkg/http/*_webhook_trigger.go` |
269+
| Change version matching logic | `internal/policy/` |
270+
| Modify K8s update behavior | `provider/kubernetes/` |
271+
| Add notification channel | `extension/notification/` |
272+
| Change polling behavior | `trigger/poll/` |
273+
| Modify approval workflow | `approvals/`, `bot/` |
274+
| Add registry authentication | `extension/credentialshelper/` |
275+
| Parse image references | `util/` |
276+
| HTTP API endpoints | `pkg/http/` |
277+
278+
## Testing
279+
280+
```bash
281+
# Unit tests
282+
make test
283+
284+
# E2E tests (requires running cluster)
285+
make e2e
286+
```
287+
288+
Test files follow Go convention: `*_test.go` alongside source files.
289+
290+
## Frontend (UI)
291+
292+
The web dashboard is a Vue.js application in `ui/`:
293+
294+
```bash
295+
cd ui
296+
yarn install
297+
yarn run serve # Development
298+
yarn run build # Production build
299+
```
300+
301+
Built assets go to `ui/dist/`, served by Keel's HTTP server.

0 commit comments

Comments
 (0)