Add reference documentation for Databuddy SDK and core packages

Co-authored-by: eassanassar <[email protected]>

Author: cursoragent

docs/reference/README.md

@@ -0,0 +1,11 @@
+## Reference Documentation
+
+- **SDK**: [sdk.md](./sdk.md)
+- **HTTP API**: [api.md](./api.md)
+- **Validation**: [validation.md](./validation.md)
+- **Shared Utilities**: [shared.md](./shared.md)
+- **RPC**: [rpc.md](./rpc.md)
+- **Auth**: [auth.md](./auth.md)
+- **Redis & Caching**: [redis.md](./redis.md)
+- **Database**: [db.md](./db.md)
+- **Event Mapper**: [mapper.md](./mapper.md)

docs/reference/api.md

@@ -0,0 +1,101 @@
+## HTTP API
+
+Base URL examples:
+- Production: `https://api.databuddy.cc`
+- Local dev: your `apps/api` server (default port 3001)
+
+Authentication:
+- Most routes require either a logged-in session cookie or an `x-api-key` header scoped to the website.
+
+### Health
+- **GET** `/health`
+- Returns service status for ClickHouse, Database, and Redis.
+- Example:
+```bash
+curl -s https://api.databuddy.cc/health | jq
+```
+
+### Query API (analytics)
+- **Prefix**: `/v1/query`
+- Applies website authentication middleware. If `website_id` is required and the site is not public, you must be authenticated (session or `x-api-key`).
+
+- **GET** `/v1/query/types?include_meta=true`
+  - Returns available query types and configuration (allowed filters, defaults, optional meta).
+  - Example:
+```bash
+curl -s "https://api.databuddy.cc/v1/query/types?include_meta=true" \
+  -H "cookie: ...session..." | jq
+```
+
+- **POST** `/v1/query/compile`
+  - Validates and compiles a query request without executing it.
+  - Body fields:
+    - `projectId` (string), `type` (string), `from` (ISO date), `to` (ISO date)
+    - Optional: `timeUnit`, `filters[]`, `groupBy[]`, `orderBy`, `limit`, `offset`
+  - Example:
+```bash
+curl -s -X POST https://api.databuddy.cc/v1/query/compile?website_id=WEBSITE_ID \
+  -H "content-type: application/json" \
+  -H "cookie: ...session..." \
+  -d '{
+    "projectId": "WEBSITE_ID",
+    "type": "summary_overview",
+    "from": "2025-01-01",
+    "to": "2025-01-31",
+    "timeUnit": "day",
+    "limit": 100
+  }' | jq
+```
+
+- **POST** `/v1/query`
+  - Executes one or more dynamic query requests. Supports batch input (array).
+  - Body (single request):
+    - `parameters`: Array of strings or objects `{ name, start_date?, end_date?, granularity?, id? }`
+    - Optional: `limit`, `page`, `filters[]`, `granularity`, `groupBy`, `startDate`, `endDate`, `timeZone`
+  - Example (single):
+```bash
+curl -s -X POST "https://api.databuddy.cc/v1/query?website_id=WEBSITE_ID" \
+  -H "content-type: application/json" \
+  -H "x-api-key: YOUR_WEBSITE_API_KEY" \
+  -d '{
+    "parameters": ["summary_overview"],
+    "limit": 50,
+    "granularity": "daily"
+  }' | jq
+```
+  - Example (batch):
+```bash
+curl -s -X POST "https://api.databuddy.cc/v1/query?website_id=WEBSITE_ID" \
+  -H "content-type: application/json" \
+  -H "cookie: ...session..." \
+  -d '[
+    { "parameters": ["summary_overview"], "granularity": "daily" },
+    { "parameters": [{ "name": "pages_top", "start_date": "2025-01-01", "end_date": "2025-01-31" }] }
+  ]' | jq
+```
+
+### Assistant API (streaming)
+- **Prefix**: `/v1/assistant`
+- **POST** `/v1/assistant/stream`
+  - Streams assistant responses for a given website and message.
+  - Body: `{ message: string, website_id: string, model?: 'chat'|'agent'|'agent-max', context?: { previousMessages?: { role?: string, content: string }[] } }`
+  - Example (fetch streaming):
+```ts
+async function streamAssistant() {
+  const res = await fetch('https://api.databuddy.cc/v1/assistant/stream?website_id=WEBSITE_ID', {
+    method: 'POST',
+    headers: { 'content-type': 'application/json', 'cookie': '...session...' },
+    body: JSON.stringify({ message: 'Show sessions trend', website_id: 'WEBSITE_ID' })
+  });
+  const reader = res.body!.getReader();
+  const decoder = new TextDecoder();
+  while (true) {
+    const { value, done } = await reader.read();
+    if (done) break;
+    console.log(decoder.decode(value)); // streaming updates
+  }
+}
+```
+
+### Authentication Notes
+- Routes enforce website access using either session cookies or `x-api-key` with appropriate scope. Public websites bypass auth for read access.

docs/reference/auth.md

@@ -0,0 +1,33 @@
+## @databuddy/auth
+
+### Overview
+- Wraps `better-auth` with Drizzle adapter, email providers, OTP, magic links, organizations, and custom session fields.
+- Exports:
+  - `auth`: Better Auth instance (`auth.api.getSession({ headers })`, etc.)
+  - `websitesApi`: `{ hasPermission: auth.api.hasPermission }`
+  - Types: `User`, `Session`
+  - Permissions/roles helpers
+
+### Getting the current session (server)
+```ts
+import { auth } from '@databuddy/auth';
+
+const session = await auth.api.getSession({ headers: request.headers });
+if (!session?.user) {
+  // not authenticated
+}
+```
+
+### Checking website permissions
+```ts
+import { websitesApi } from '@databuddy/auth';
+
+const { success } = await websitesApi.hasPermission({
+  headers: request.headers,
+  body: { permissions: { website: ['read'] } },
+});
+```
+
+### Notes
+- The package configures email delivery using Resend and sends verification, magic link, OTP, and invitation emails.
+- Sessions are cached with Redis for performance.

docs/reference/db.md

@@ -0,0 +1,18 @@
+## @databuddy/db
+
+### Overview
+- Exposes Drizzle ORM (`export * from 'drizzle-orm'`), a configured Postgres client `db`, and ClickHouse client/schema exports.
+- Also exports all Drizzle `schema` and `relations` so you can write typed queries.
+
+### Usage
+```ts
+import { db, eq, websites } from '@databuddy/db';
+
+const list = await db.query.websites.findMany({
+  where: eq(websites.userId, userId),
+  limit: 20,
+});
+```
+
+### Environment
+- Requires `DATABASE_URL` to be set.

docs/reference/mapper.md

@@ -0,0 +1,18 @@
+## @databuddy/mapper
+
+### Overview
+- Normalizes third-party analytics export rows into Databuddy `AnalyticsEvent` objects.
+- Exports:
+  - `mapEvents(adapter, rows)`
+  - `adapters.umami(clientId)` and `AnalyticsEventAdapter` type
+
+### Umami Adapter
+```ts
+import { adapters, mapEvents } from '@databuddy/mapper';
+
+const adapter = adapters.umami('YOUR_CLIENT_ID');
+const events = mapEvents(adapter, umamiCsvRows);
+// events: AnalyticsEvent[] ready for ingestion
+```
+
+The adapter maps fields like `session_id`, `distinct_id`, geo, screen, UTM fields, page title, and timestamps to Databuddy event structure.

docs/reference/redis.md

@@ -0,0 +1,72 @@
+## @databuddy/redis
+
+### Redis Client
+- `redis`: a singleton ioredis client (JSON helpers included)
+- `getRedisCache()`: returns the singleton client (throws if `REDIS_URL` missing)
+- `getRawRedis()`: returns a non-extended raw client
+- `getJson<T>(key): Promise<T|null>` and `setJson<T>(key, value, expireInSec)` on the extended client
+- Lock helpers: `getLock(key, value, timeoutMs)`, `releaseLock(key, value)`, `isRedisConnected()`, `disconnectRedis()`
+
+```ts
+import { redis, getLock, releaseLock } from '@databuddy/redis';
+
+await redis.set('hello', 'world');
+const got = await redis.get('hello');
+
+const ok = await getLock('my-lock', 'job-123', 10_000);
+if (ok) {
+  try {
+    // do work
+  } finally {
+    await releaseLock('my-lock', 'job-123');
+  }
+}
+```
+
+### cacheable(fn, options)
+- Memoizes async functions in Redis with SWR support.
+- Options: `expireInSec`, `prefix?`, `serialize?`, `deserialize?`, `staleWhileRevalidate?`, `staleTime?`, `maxRetries?`.
+- Returns a wrapped function with helpers: `.getKey(...)`, `.clear(...)`, `.clearAll()`, `.invalidate(...)`.
+
+```ts
+import { cacheable } from '@databuddy/redis';
+
+const getUser = cacheable(async (id: string) => {
+  // fetch from DB
+  return { id, name: 'Ada' };
+}, { expireInSec: 300, prefix: 'user', staleWhileRevalidate: true, staleTime: 30 });
+
+const u = await getUser('123');
+await getUser.clear('123');
+```
+
+### getCache(key, options, fn)
+- Low-level helper to read-through cache a promise with SWR.
+
+```ts
+import { getCache } from '@databuddy/redis';
+
+const data = await getCache('stats:today', { expireInSec: 60 }, async () => {
+  // compute expensive stats
+  return { sessions: 42 };
+});
+```
+
+### Drizzle Cache
+- `createDrizzleCache({ redis, namespace? })` provides:
+  - `withCache({ key, ttl?, tables?, tag?, autoInvalidate?, queryFn })`
+  - `invalidateByTables(tables: string[])`, `invalidateByTags(tags: string[])`, `invalidateByKey(key: string)`
+  - `cleanupDeps()`
+
+Example:
+```ts
+import { createDrizzleCache, redis } from '@databuddy/redis';
+
+const websiteCache = createDrizzleCache({ redis, namespace: 'websites' });
+const site = await websiteCache.withCache({
+  key: 'getById:abc',
+  ttl: 600,
+  tables: ['websites'],
+  queryFn: () => db.query.websites.findFirst({ where: eq(websites.id, 'abc') })
+});
+```

docs/reference/rpc.md

@@ -0,0 +1,93 @@
+## @databuddy/rpc
+
+### Overview
+- Uses tRPC for type-safe server procedures.
+- Exports:
+  - `appRouter`, `type AppRouter`
+  - `createTRPCContext`, `createTRPCRouter`, `publicProcedure`, `protectedProcedure`, `rateLimitedProtectedProcedure`, `rateLimitedAdminProcedure`
+  - `getRateLimitIdentifier`, `rateLimiters`
+
+### Context
+```ts
+import { createTRPCContext } from '@databuddy/rpc';
+
+// Called per request (headers required)
+const ctx = await createTRPCContext({ headers: request.headers });
+// ctx = { db, auth, session, user, headers }
+```
+
+### Routers
+```ts
+import { createTRPCRouter, publicProcedure, protectedProcedure } from '@databuddy/rpc';
+
+export const exampleRouter = createTRPCRouter({
+  hello: publicProcedure.query(() => 'world'),
+  me: protectedProcedure.query(({ ctx }) => ctx.user),
+});
+```
+
+The `appRouter` mounts feature routers like `websites`, `funnels`, `preferences`, `goals`, `autocomplete`, `apikeys`, `experiments`.
+
+### Rate Limiting Utilities
+- `getRateLimitIdentifier(userId?: string, headers?: Headers): string`
+- `RateLimiter` class with methods:
+  - `checkLimit(identifier: string): Promise<{ success; limit; remaining; reset; }>`
+  - `getStatus(identifier: string)`
+  - `reset(identifier: string)`
+- Built-in `rateLimiters`: `api`, `auth`, `expensive`, `admin`, `public`.
+
+Use within procedures via `rateLimitedProtectedProcedure` or in HTTP middleware (see `apps/api`).
+
+### Referrer Utilities
+- `parseReferrer(referrerUrl: string | null | undefined, currentDomain?: string): { type; name; url; domain }`
+- `categorizeReferrer(info): string`
+- `isInternalReferrer(referrerUrl: string, websiteHostname?: string): boolean`
+
+Example:
+```ts
+import { parseReferrer, categorizeReferrer } from '@databuddy/rpc/utils/referrer';
+
+const info = parseReferrer('https://www.google.com/search?q=databuddy', 'example.com');
+// { type: 'search', name: 'www.google.com', ... }
+const category = categorizeReferrer(info); // 'Search Engine'
+```
+
+### Auth Utilities
+- `authorizeWebsiteAccess(ctx, websiteId, permission: 'read'|'update'|'delete'|'transfer')`
+  - Ensures the current user (from tRPC context) has access to the given website, considering public sites, owner, admin, or org permissions; throws `TRPCError` otherwise.
+```ts
+import { authorizeWebsiteAccess } from '@databuddy/rpc/utils/auth';
+
+export const websitesRouter = createTRPCRouter({
+  get: protectedProcedure.input(z.string()).query(async ({ ctx, input }) => {
+    const site = await authorizeWebsiteAccess(ctx, input, 'read');
+    return site;
+  })
+});
+```
+
+### Billing Utilities
+- `checkAndTrackWebsiteCreation(customerId: string)` → `{ allowed: boolean, error?: string }`
+- `trackWebsiteUsage(customerId: string, value: number)` → `{ success: boolean }`
+- `getBillingCustomerId(userId: string, organizationId?: string|null): Promise<string>`
+
+```ts
+import { getBillingCustomerId, checkAndTrackWebsiteCreation } from '@databuddy/rpc/utils/billing';
+
+const customerId = await getBillingCustomerId(ctx.user.id, ctx.user.organizationId);
+const { allowed } = await checkAndTrackWebsiteCreation(customerId);
+if (!allowed) throw new TRPCError({ code: 'FORBIDDEN' });
+```
+
+### Cache Invalidation Helpers
+- `invalidateBasicWebsiteCaches(websiteId, websiteCache)`
+- `invalidateWebsiteCaches(websiteId, userId, reason?)`
+
+```ts
+import { createDrizzleCache, redis } from '@databuddy/redis';
+import { invalidateBasicWebsiteCaches, invalidateWebsiteCaches } from '@databuddy/rpc/utils/cache-invalidation';
+
+const websiteCache = createDrizzleCache({ redis, namespace: 'websites' });
+await invalidateBasicWebsiteCaches(websiteId, websiteCache);
+await invalidateWebsiteCaches(websiteId, ctx.user.id, 'website updated');
+```