Skip to content

Commit 653c220

Browse files
authored
Merge pull request #286 from cipherstash/stack-package
feat: add CipherStash Stack package with encryption, secrets, and int…
2 parents e34838f + be304dc commit 653c220

File tree

183 files changed

+30610
-1226
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

183 files changed

+30610
-1226
lines changed
Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
---
2+
name: create-evlog-adapter
3+
description: Create a new built-in evlog adapter to send wide events to an external observability platform. Use when adding a new drain adapter (e.g., for Datadog, Sentry, Loki, Elasticsearch, etc.) to the evlog package. Covers source code, build config, package exports, tests, and all documentation.
4+
---
5+
6+
# Create evlog Adapter
7+
8+
Add a new built-in adapter to evlog. Every adapter follows the same architecture. This skill walks through all 8 touchpoints. **Every single touchpoint is mandatory** -- do not skip any.
9+
10+
## PR Title
11+
12+
Recommended format for the pull request title:
13+
14+
```
15+
feat: add {name} adapter
16+
```
17+
18+
The exact wording may vary depending on the adapter (e.g., `feat: add OTLP adapter`, `feat: add Axiom drain adapter`), but it should always follow the `feat:` conventional commit prefix.
19+
20+
## Touchpoints Checklist
21+
22+
| # | File | Action |
23+
|---|------|--------|
24+
| 1 | `packages/evlog/src/adapters/{name}.ts` | Create adapter source |
25+
| 2 | `packages/evlog/tsdown.config.ts` | Add build entry |
26+
| 3 | `packages/evlog/package.json` | Add `exports` + `typesVersions` entries |
27+
| 4 | `packages/evlog/test/adapters/{name}.test.ts` | Create tests |
28+
| 5 | `apps/docs/content/3.adapters/{n}.{name}.md` | Create adapter doc page (before `custom.md`) |
29+
| 6 | `apps/docs/content/3.adapters/1.overview.md` | Add adapter to overview (links, card, env vars) |
30+
| 7 | `AGENTS.md` | Add adapter to the "Built-in Adapters" table |
31+
| 8 | Renumber `custom.md` | Ensure `custom.md` stays last after the new adapter |
32+
33+
**Important**: Do NOT consider the task complete until all 8 touchpoints have been addressed.
34+
35+
## Naming Conventions
36+
37+
Use these placeholders consistently:
38+
39+
| Placeholder | Example (Datadog) | Usage |
40+
|-------------|-------------------|-------|
41+
| `{name}` | `datadog` | File names, import paths, env var suffix |
42+
| `{Name}` | `Datadog` | PascalCase in function/interface names |
43+
| `{NAME}` | `DATADOG` | SCREAMING_CASE in env var prefixes |
44+
45+
## Step 1: Adapter Source
46+
47+
Create `packages/evlog/src/adapters/{name}.ts`.
48+
49+
Read [references/adapter-template.md](references/adapter-template.md) for the full annotated template.
50+
51+
Key architecture rules:
52+
53+
1. **Config interface** -- service-specific fields (API key, endpoint, etc.) plus optional `timeout?: number`
54+
2. **`getRuntimeConfig()`** -- import from `./_utils` (shared helper, do NOT redefine locally)
55+
3. **Config priority** (highest to lowest):
56+
- Overrides passed to `create{Name}Drain()`
57+
- `runtimeConfig.evlog.{name}`
58+
- `runtimeConfig.{name}`
59+
- Environment variables: `NUXT_{NAME}_*` then `{NAME}_*`
60+
4. **Factory function** -- `create{Name}Drain(overrides?: Partial<Config>)` returns `(ctx: DrainContext) => Promise<void>`
61+
5. **Exported send functions** -- `sendTo{Name}(event, config)` and `sendBatchTo{Name}(events, config)` for direct use and testability
62+
6. **Error handling** -- try/catch with `console.error('[evlog/{name}] ...')`, never throw from the drain
63+
7. **Timeout** -- `AbortController` with 5000ms default, configurable via `config.timeout`
64+
8. **Event transformation** -- if the service needs a specific format, export a `to{Name}Event()` converter
65+
66+
## Step 2: Build Config
67+
68+
Add a build entry in `packages/evlog/tsdown.config.ts` alongside the existing adapters:
69+
70+
```typescript
71+
'adapters/{name}': 'src/adapters/{name}.ts',
72+
```
73+
74+
Place it after the last adapter entry (currently `sentry` at line 22).
75+
76+
## Step 3: Package Exports
77+
78+
In `packages/evlog/package.json`, add two entries:
79+
80+
**In `exports`** (after the last adapter, currently `./posthog`):
81+
82+
```json
83+
"./{name}": {
84+
"types": "./dist/adapters/{name}.d.mts",
85+
"import": "./dist/adapters/{name}.mjs"
86+
}
87+
```
88+
89+
**In `typesVersions["*"]`** (after the last adapter):
90+
91+
```json
92+
"{name}": [
93+
"./dist/adapters/{name}.d.mts"
94+
]
95+
```
96+
97+
## Step 4: Tests
98+
99+
Create `packages/evlog/test/adapters/{name}.test.ts`.
100+
101+
Read [references/test-template.md](references/test-template.md) for the full annotated template.
102+
103+
Required test categories:
104+
105+
1. URL construction (default + custom endpoint)
106+
2. Headers (auth, content-type, service-specific)
107+
3. Request body format (JSON structure matches service API)
108+
4. Error handling (non-OK responses throw with status)
109+
5. Batch operations (`sendBatchTo{Name}`)
110+
6. Timeout handling (default 5000ms + custom)
111+
112+
## Step 5: Adapter Documentation Page
113+
114+
Create `apps/docs/content/3.adapters/{n}.{name}.md` where `{n}` is the next number before `custom.md` (custom should always be last).
115+
116+
Use the existing Axiom adapter page (`apps/docs/content/3.adapters/2.axiom.md`) as a reference for frontmatter structure, tone, and sections. Key sections: intro, quick setup, configuration (env vars table + priority), advanced usage, querying in the target service, troubleshooting, direct API usage, next steps.
117+
118+
## Step 6: Update Adapters Overview Page
119+
120+
Edit `apps/docs/content/3.adapters/1.overview.md` to add the new adapter in **three** places (follow the pattern of existing adapters):
121+
122+
1. **Frontmatter `links` array** -- add a link entry with icon and path
123+
2. **`::card-group` section** -- add a card block before the Custom card
124+
3. **Zero-Config Setup `.env` example** -- add the adapter's env vars
125+
126+
## Step 7: Update AGENTS.md
127+
128+
In the root `AGENTS.md` file, "Log Draining & Adapters" section:
129+
130+
1. Add a row to the **"Built-in Adapters"** table
131+
2. Add a **"Using {Name} Adapter"** usage example block with `create{Name}Drain()` and env vars
132+
133+
Follow the pattern of existing adapters in the file.
134+
135+
## Step 8: Renumber `custom.md`
136+
137+
If the new adapter's number conflicts with `custom.md`, renumber `custom.md` to be the last entry. For example, if the new adapter is `5.{name}.md`, rename `5.custom.md` to `6.custom.md`.
138+
139+
## Verification
140+
141+
After completing all steps, run:
142+
143+
```bash
144+
cd packages/evlog
145+
bun run build # Verify build succeeds with new entry
146+
bun run test # Verify tests pass
147+
```
Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Adapter Source Template
2+
3+
Complete TypeScript template for `packages/evlog/src/adapters/{name}.ts`.
4+
5+
Replace `{Name}`, `{name}`, and `{NAME}` with the actual service name.
6+
7+
```typescript
8+
import type { DrainContext, WideEvent } from '../types'
9+
import { getRuntimeConfig } from './_utils'
10+
11+
// --- 1. Config Interface ---
12+
// Define all service-specific configuration fields.
13+
// Always include optional `timeout`.
14+
export interface {Name}Config {
15+
/** {Name} API key / token */
16+
apiKey: string
17+
/** {Name} API endpoint. Default: https://api.{name}.com */
18+
endpoint?: string
19+
/** Request timeout in milliseconds. Default: 5000 */
20+
timeout?: number
21+
// Add service-specific fields here (dataset, project, region, etc.)
22+
}
23+
24+
// --- 2. Event Transformation (optional) ---
25+
// Export a converter if the service needs a specific format.
26+
// This makes the transformation testable independently.
27+
28+
/** {Name} event structure */
29+
export interface {Name}Event {
30+
// Define the target service's event shape
31+
timestamp: string
32+
level: string
33+
data: Record<string, unknown>
34+
}
35+
36+
/**
37+
* Convert a WideEvent to {Name}'s event format.
38+
*/
39+
export function to{Name}Event(event: WideEvent): {Name}Event {
40+
const { timestamp, level, ...rest } = event
41+
42+
return {
43+
timestamp,
44+
level,
45+
data: rest,
46+
}
47+
}
48+
49+
// --- 3. Factory Function ---
50+
// Returns a drain function that resolves config at call time.
51+
// Config priority: overrides > runtimeConfig.evlog.{name} > runtimeConfig.{name} > env vars
52+
53+
/**
54+
* Create a drain function for sending logs to {Name}.
55+
*
56+
* Configuration priority (highest to lowest):
57+
* 1. Overrides passed to create{Name}Drain()
58+
* 2. runtimeConfig.evlog.{name}
59+
* 3. runtimeConfig.{name}
60+
* 4. Environment variables: NUXT_{NAME}_*, {NAME}_*
61+
*
62+
* @example
63+
* ```ts
64+
* // Zero config - set NUXT_{NAME}_API_KEY env var
65+
* nitroApp.hooks.hook('evlog:drain', create{Name}Drain())
66+
*
67+
* // With overrides
68+
* nitroApp.hooks.hook('evlog:drain', create{Name}Drain({
69+
* apiKey: 'my-key',
70+
* }))
71+
* ```
72+
*/
73+
export function create{Name}Drain(overrides?: Partial<{Name}Config>): (ctx: DrainContext) => Promise<void> {
74+
return async (ctx: DrainContext) => {
75+
const runtimeConfig = getRuntimeConfig()
76+
const evlogConfig = runtimeConfig?.evlog?.{name}
77+
const rootConfig = runtimeConfig?.{name}
78+
79+
// Build config with fallbacks
80+
const config: Partial<{Name}Config> = {
81+
apiKey: overrides?.apiKey ?? evlogConfig?.apiKey ?? rootConfig?.apiKey
82+
?? process.env.NUXT_{NAME}_API_KEY ?? process.env.{NAME}_API_KEY,
83+
endpoint: overrides?.endpoint ?? evlogConfig?.endpoint ?? rootConfig?.endpoint
84+
?? process.env.NUXT_{NAME}_ENDPOINT ?? process.env.{NAME}_ENDPOINT,
85+
timeout: overrides?.timeout ?? evlogConfig?.timeout ?? rootConfig?.timeout,
86+
}
87+
88+
// Validate required fields
89+
if (!config.apiKey) {
90+
console.error('[evlog/{name}] Missing apiKey. Set NUXT_{NAME}_API_KEY env var or pass to create{Name}Drain()')
91+
return
92+
}
93+
94+
try {
95+
await sendTo{Name}(ctx.event, config as {Name}Config)
96+
} catch (error) {
97+
console.error('[evlog/{name}] Failed to send event:', error)
98+
}
99+
}
100+
}
101+
102+
// --- 5. Send Functions ---
103+
// Exported for direct use and testability.
104+
// sendTo{Name} wraps sendBatchTo{Name} for single events.
105+
106+
/**
107+
* Send a single event to {Name}.
108+
*/
109+
export async function sendTo{Name}(event: WideEvent, config: {Name}Config): Promise<void> {
110+
await sendBatchTo{Name}([event], config)
111+
}
112+
113+
/**
114+
* Send a batch of events to {Name}.
115+
*/
116+
export async function sendBatchTo{Name}(events: WideEvent[], config: {Name}Config): Promise<void> {
117+
if (events.length === 0) return
118+
119+
const endpoint = (config.endpoint ?? 'https://api.{name}.com').replace(/\/$/, '')
120+
const timeout = config.timeout ?? 5000
121+
// Construct the full URL for the service's ingest API
122+
const url = `${endpoint}/v1/ingest`
123+
124+
const headers: Record<string, string> = {
125+
'Content-Type': 'application/json',
126+
'Authorization': `Bearer ${config.apiKey}`,
127+
// Add service-specific headers here
128+
}
129+
130+
// Transform events if the service needs a specific format
131+
const payload = events.map(to{Name}Event)
132+
// Or send raw: JSON.stringify(events)
133+
134+
const controller = new AbortController()
135+
const timeoutId = setTimeout(() => controller.abort(), timeout)
136+
137+
try {
138+
const response = await fetch(url, {
139+
method: 'POST',
140+
headers,
141+
body: JSON.stringify(payload),
142+
signal: controller.signal,
143+
})
144+
145+
if (!response.ok) {
146+
const text = await response.text().catch(() => 'Unknown error')
147+
const safeText = text.length > 200 ? `${text.slice(0, 200)}...[truncated]` : text
148+
throw new Error(`{Name} API error: ${response.status} ${response.statusText} - ${safeText}`)
149+
}
150+
} finally {
151+
clearTimeout(timeoutId)
152+
}
153+
}
154+
```
155+
156+
## Customization Notes
157+
158+
- **Auth style**: Some services use `Authorization: Bearer`, others use a custom header like `X-API-Key`. Adjust the headers accordingly.
159+
- **Payload format**: Some services accept raw JSON arrays (Axiom), others need a wrapper object (PostHog `{ api_key, batch }`), others need a protocol-specific structure (OTLP). Adapt `sendBatchTo{Name}` to match.
160+
- **Event transformation**: If the service expects a specific schema, implement `to{Name}Event()`. If the service accepts arbitrary JSON, you can skip it and send `ctx.event` directly.
161+
- **URL construction**: Match the service's API endpoint pattern. Some use path-based routing (`/v1/datasets/{id}/ingest`), others use a flat endpoint (`/batch/`).
162+
- **Extra config fields**: Add service-specific fields to the config interface (e.g., `dataset` for Axiom, `orgId` for org-scoped APIs, `host` for region selection).

0 commit comments

Comments
 (0)