Skip to content

Commit 4f9b6b4

Browse files
titotito
authored
feat: add credential substitution with global credentials support (#26)
## Summary End-to-end credential substitution: real API keys never reach sandboxed processes. The proxy transparently replaces opaque placeholders with real values before forwarding HTTP requests upstream. ### Global credentials Global credentials are stored in the dashboard and injected on demand via `--inject`: ```bash greywall --inject ANTHROPIC_API_KEY --inject OPENAI_API_KEY -- opencode ``` The session creation API (`POST /api/sessions`) accepts a `global_credentials` field (list of labels). The proxy resolves each label, merges placeholder-to-value mappings into the session, and returns the placeholders so greywall can set them as environment variables. **Example request:** ```json { "session_id": "gw-abc123", "container_name": "opencode", "global_credentials": ["ANTHROPIC_API_KEY"], "ttl_seconds": 900 } ``` **Example response:** ```json { "session_id": "gw-abc123", "expires_at": "2026-03-25T23:00:00Z", "credential_count": 1, "global_credentials": { "ANTHROPIC_API_KEY": "greyproxy:credential:v1:global:a1b2c3..." } } ``` ### Substitution tracking - Substitution counts flushed to DB every 60s, broadcast via WebSocket (`session.substitution` event) - Activity and traffic tables show shield icon for substituted requests, credential labels as badges in expanded details - Session cards show creation time and active duration ### Changes - **API**: `POST /api/sessions` accepts `global_credentials`, resolves labels, returns placeholders - **Credential store**: publishes `session.substitution` events after flushing counts - **Activity**: `ActivityItem` includes `SubstitutedCredentials`; shield icon and label badges in both activity and traffic tables - **Settings UI**: global credentials section explains `--inject` workflow and substitution behavior - **Tests**: CRUD, API handler, and end-to-end substitution tests for global credentials - **Docs**: `docs/credential-substitution.md` covering the full API, substitution behavior, and dashboard UI ## Test plan - [x] `go test ./internal/greyproxy/...` passes - [x] `go test ./internal/greyproxy/api/...` passes (session creation tests) - [x] `greywall --inject LABEL -- env` shows placeholder in output - [x] HTTP request with placeholder substituted (shield icon in activity view) - [x] Settings > Credentials > Active Sessions shows substitution count updating ## Dependencies Companion greywall PR: GreyhavenHQ/greywall#63
1 parent 69976be commit 4f9b6b4

26 files changed

+3819
-184
lines changed

‎cmd/greyproxy/program.go‎

Lines changed: 61 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import (
1515
"runtime"
1616
"strconv"
1717
"strings"
18+
"time"
1819
"syscall"
1920

2021
"github.com/andybalholm/brotli"
@@ -43,8 +44,9 @@ type program struct {
4344
srvGreyproxy *greyproxy.Service
4445
srvProfiling *http.Server
4546

46-
cancel context.CancelFunc
47-
assemblerCancel context.CancelFunc
47+
cancel context.CancelFunc
48+
assemblerCancel context.CancelFunc
49+
credStoreCancel context.CancelFunc
4850
}
4951

5052
func (p *program) initParser() {
@@ -204,6 +206,9 @@ func (p *program) Stop(s service.Service) error {
204206
p.srvProfiling.Close()
205207
logger.Default().Debug("service @profiling shutdown")
206208
}
209+
if p.credStoreCancel != nil {
210+
p.credStoreCancel()
211+
}
207212
if p.assemblerCancel != nil {
208213
p.assemblerCancel()
209214
}
@@ -322,6 +327,44 @@ func (p *program) buildGreyproxyService() error {
322327
gostx.SetGlobalMitmEnabled(enabled)
323328
})
324329

330+
// Initialize credential substitution encryption key and store
331+
encKey, newKey, err := greyproxy.LoadOrGenerateKey(greyproxyDataHome())
332+
if err != nil {
333+
log.Warnf("credential substitution disabled: %v", err)
334+
} else {
335+
shared.EncryptionKey = encKey
336+
credStore, err := greyproxy.NewCredentialStore(shared.DB, encKey, shared.Bus)
337+
if err != nil {
338+
log.Warnf("credential store init failed: %v", err)
339+
} else {
340+
shared.CredentialStore = credStore
341+
if newKey {
342+
if sessions, globals, err := credStore.PurgeUnreadableCredentials(); err == nil && (sessions > 0 || globals > 0) {
343+
log.Infof("purged %d sessions and %d global credentials (new encryption key)", sessions, globals)
344+
}
345+
}
346+
credStoreCtx, credStoreCancel := context.WithCancel(context.Background())
347+
p.credStoreCancel = credStoreCancel
348+
credStore.StartCleanupLoop(credStoreCtx, 60*time.Second)
349+
// Wire credential substitution into the MITM pipeline
350+
gostx.SetGlobalCredentialSubstituter(func(req *http.Request) *gostx.CredentialSubstitutionInfo {
351+
result := credStore.SubstituteRequest(req)
352+
if result.Count == 0 {
353+
return nil
354+
}
355+
var sessionID string
356+
if len(result.SessionIDs) > 0 {
357+
sessionID = result.SessionIDs[0]
358+
}
359+
return &gostx.CredentialSubstitutionInfo{
360+
Labels: result.Labels,
361+
SessionID: sessionID,
362+
}
363+
})
364+
log.Infof("credential store loaded: %d mappings from %d sessions", credStore.Size(), credStore.SessionCount())
365+
}
366+
}
367+
325368
shared.Version = version
326369

327370
// Collect listening ports for the health endpoint
@@ -375,20 +418,22 @@ func (p *program) buildGreyproxyService() error {
375418
redactedRespHeaders := redactor.Redact(info.ResponseHeaders)
376419

377420
txn, err := greyproxy.CreateHttpTransaction(shared.DB, greyproxy.HttpTransactionCreateInput{
378-
ContainerName: containerName,
379-
DestinationHost: host,
380-
DestinationPort: port,
381-
Method: info.Method,
382-
URL: "https://" + info.Host + info.URI,
383-
RequestHeaders: redactedReqHeaders,
384-
RequestBody: reqBody,
385-
RequestContentType: reqCT,
386-
StatusCode: info.StatusCode,
387-
ResponseHeaders: redactedRespHeaders,
388-
ResponseBody: respBody,
389-
ResponseContentType: respCT,
390-
DurationMs: info.DurationMs,
391-
Result: "auto",
421+
ContainerName: containerName,
422+
DestinationHost: host,
423+
DestinationPort: port,
424+
Method: info.Method,
425+
URL: "https://" + info.Host + info.URI,
426+
RequestHeaders: redactedReqHeaders,
427+
RequestBody: reqBody,
428+
RequestContentType: reqCT,
429+
StatusCode: info.StatusCode,
430+
ResponseHeaders: redactedRespHeaders,
431+
ResponseBody: respBody,
432+
ResponseContentType: respCT,
433+
DurationMs: info.DurationMs,
434+
Result: "auto",
435+
SubstitutedCredentials: info.SubstitutedCredentials,
436+
SessionID: info.SessionID,
392437
})
393438
if err != nil {
394439
log.Warnf("failed to store HTTP transaction: %v", err)

‎docs/credential-substitution-api.md‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# Credential Substitution API
2+
3+
This document describes the REST API that greywall (or any sandbox client) uses to register credential substitution sessions with greyproxy.
4+
5+
## Overview
6+
7+
When greywall launches a sandboxed process, it:
8+
9+
1. Reads the process's environment variables for sensitive values (API keys, tokens, etc.)
10+
2. Generates opaque placeholder strings for each credential
11+
3. Passes the placeholders to the sandboxed process (via modified env vars)
12+
4. Registers a session with greyproxy, providing the placeholder-to-real-value mappings
13+
14+
GreyProxy then transparently replaces placeholders with real credentials in HTTP headers and query parameters before forwarding requests upstream.
15+
16+
## Session Lifecycle
17+
18+
### Create or Update Session
19+
20+
```
21+
POST /api/sessions
22+
Content-Type: application/json
23+
```
24+
25+
**Request body:**
26+
27+
```json
28+
{
29+
"session_id": "uuid-string",
30+
"container_name": "opencode",
31+
"mappings": {
32+
"greyproxy:credential:v1:SESSION_ID:HEX": "sk-real-api-key-value",
33+
"greyproxy:credential:v1:SESSION_ID:HEX2": "another-real-key"
34+
},
35+
"labels": {
36+
"greyproxy:credential:v1:SESSION_ID:HEX": "OPENAI_API_KEY",
37+
"greyproxy:credential:v1:SESSION_ID:HEX2": "ANTHROPIC_API_KEY"
38+
},
39+
"metadata": {
40+
"pwd": "/home/user/project",
41+
"cmd": "opencode",
42+
"args": "--model claude-sonnet-4-20250514",
43+
"binary_path": "/usr/bin/opencode",
44+
"pid": "12345"
45+
},
46+
"ttl_seconds": 900
47+
}
48+
```
49+
50+
**Fields:**
51+
52+
| Field | Type | Required | Description |
53+
|---|---|---|---|
54+
| `session_id` | string | Yes | Unique session identifier (UUID recommended). Used for upserts. |
55+
| `container_name` | string | Yes | Name of the sandboxed container/process. Used for log correlation. |
56+
| `mappings` | map[string]string | Yes | Placeholder string to real credential value. Keys must use the `greyproxy:credential:` prefix format. |
57+
| `labels` | map[string]string | No | Placeholder string to human-readable label (e.g. env var name). Same keys as `mappings`. |
58+
| `metadata` | map[string]string | No | Arbitrary key-value metadata about the session. Displayed in the dashboard. |
59+
| `ttl_seconds` | int | No | Session TTL in seconds (default: 900, max: 3600). |
60+
61+
**Response (200):**
62+
63+
```json
64+
{
65+
"session_id": "uuid-string",
66+
"expires_at": "2026-03-25T16:00:00Z",
67+
"credential_count": 2
68+
}
69+
```
70+
71+
### Heartbeat
72+
73+
Reset the TTL for an active session. Call this periodically to keep the session alive.
74+
75+
```
76+
POST /api/sessions/:id/heartbeat
77+
```
78+
79+
**Response (200):**
80+
81+
```json
82+
{
83+
"session_id": "uuid-string",
84+
"expires_at": "2026-03-25T16:15:00Z"
85+
}
86+
```
87+
88+
**Response (404):** Session not found or expired.
89+
90+
### Delete Session
91+
92+
Immediately expire and remove a session.
93+
94+
```
95+
DELETE /api/sessions/:id
96+
```
97+
98+
**Response (200):**
99+
100+
```json
101+
{
102+
"session_id": "uuid-string",
103+
"deleted": true
104+
}
105+
```
106+
107+
### List Sessions
108+
109+
Returns all active (non-expired) sessions.
110+
111+
```
112+
GET /api/sessions
113+
```
114+
115+
**Response (200):** Array of session objects with credential labels, counts, metadata, and timestamps.
116+
117+
## Metadata Convention
118+
119+
The `metadata` field is a flexible string map. Greywall can send any keys it finds useful. The following keys are recognized and displayed prominently in the dashboard:
120+
121+
| Key | Description | Example |
122+
|---|---|---|
123+
| `pwd` | Working directory of the sandboxed process | `/home/user/project` |
124+
| `cmd` | Command name | `opencode` |
125+
| `args` | Command arguments | `--model claude-sonnet-4-20250514` |
126+
| `binary_path` | Absolute path to the binary | `/usr/bin/opencode` |
127+
| `pid` | PID of the greywall sandbox process | `12345` |
128+
| `created_by` | What created the session | `greywall v0.2.0` |
129+
130+
## Placeholder Format
131+
132+
Placeholders follow this format:
133+
134+
```
135+
greyproxy:credential:v1:<scope>:<hex>
136+
```
137+
138+
- `v1` is the version prefix
139+
- `<scope>` is either a session ID or `"global"` for global credentials
140+
- `<hex>` is a random hex string for uniqueness
141+
142+
The client can generate these using `GeneratePlaceholder()` or construct them manually. The only requirement is that they start with `greyproxy:credential:` so the proxy's fast-path check can skip scanning headers that don't contain any placeholders.
143+
144+
## What Gets Substituted
145+
146+
The proxy scans and substitutes placeholders in:
147+
148+
- **HTTP request headers** (all header values)
149+
- **URL query parameters**
150+
151+
It does **not** substitute in:
152+
153+
- Request bodies (the body is stored as-is with placeholders visible)
154+
- Response data
155+
156+
## Transaction Tracking
157+
158+
When credentials are substituted in a request, the resulting HTTP transaction is tagged with:
159+
160+
- `substituted_credentials`: JSON array of credential label names that were substituted
161+
- `session_id`: The session that provided the credentials
162+
163+
These fields are visible in the transaction detail view and can be used for filtering via `GET /api/transactions?session_id=...`.

0 commit comments

Comments
 (0)