|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +This is **@mgcrea/prisma-queue**, a job queue library for Prisma + PostgreSQL. It leverages PostgreSQL's `SKIP LOCKED` feature for reliable job dequeuing and supports crontab syntax for scheduled jobs. |
| 8 | + |
| 9 | +## Commands |
| 10 | + |
| 11 | +### Development |
| 12 | +- `pnpm dev` - Run tests in watch mode with debug output |
| 13 | +- `pnpm start` - Run tests with full debug output |
| 14 | +- `pnpm spec` - Run all tests once |
| 15 | +- `pnpm test` - Run full test suite (lint, prettify, typecheck, spec) |
| 16 | + |
| 17 | +### Building & Type Checking |
| 18 | +- `pnpm build` - Build the library with tsup (outputs both CJS and ESM) |
| 19 | +- `pnpm typecheck` - Run TypeScript type checking |
| 20 | + |
| 21 | +### Code Quality |
| 22 | +- `pnpm lint` - Lint src/ and test/ directories |
| 23 | +- `pnpm prettycheck` - Check formatting |
| 24 | +- `pnpm prettify` - Format code |
| 25 | + |
| 26 | +### Database |
| 27 | +- `pnpm prepare` - Generate Prisma client (runs automatically after install) |
| 28 | +- `pnpm reset` - Force reset database and regenerate Prisma client |
| 29 | + |
| 30 | +### Testing |
| 31 | +- `pnpm coverage` - Run tests with coverage report |
| 32 | +- Tests use Vitest with `--pool=forks` for process isolation |
| 33 | + |
| 34 | +## Architecture |
| 35 | + |
| 36 | +### Core Classes |
| 37 | + |
| 38 | +**PrismaQueue** (`src/PrismaQueue.ts`) |
| 39 | +- Main queue class that manages job processing |
| 40 | +- Key methods: |
| 41 | + - `start()` - Begin polling and processing jobs |
| 42 | + - `stop()` - Stop processing |
| 43 | + - `enqueue(payload, options)` / `add()` - Add jobs to queue |
| 44 | + - `schedule(options, payload)` - Schedule recurring jobs with cron |
| 45 | + - `size(onlyAvailable?)` - Count pending jobs |
| 46 | +- Uses an event emitter pattern with events: `enqueue`, `dequeue`, `success`, `error` |
| 47 | +- Implements concurrency control via `maxConcurrency` option |
| 48 | +- Uses PostgreSQL `FOR UPDATE SKIP LOCKED` for reliable job locking (line 319 in PrismaQueue.ts) |
| 49 | + |
| 50 | +**PrismaJob** (`src/PrismaJob.ts`) |
| 51 | +- Represents an individual job instance |
| 52 | +- Key methods: |
| 53 | + - `progress(percentage)` - Update job progress |
| 54 | + - `update(data)` - Update job record |
| 55 | + - `delete()` - Remove job from queue |
| 56 | + - `fetch()` - Refresh job state from database |
| 57 | + - `isLocked()` - Check if job is locked by another transaction |
| 58 | + |
| 59 | +### Database Schema |
| 60 | + |
| 61 | +The `QueueJob` model (in `prisma/schema.prisma`) stores: |
| 62 | +- `queue` - Queue name for partitioning jobs |
| 63 | +- `key` + `cron` - For scheduled/recurring jobs |
| 64 | +- `payload` / `result` / `error` - Job data (JSON) |
| 65 | +- `priority` / `attempts` / `maxAttempts` - Execution control |
| 66 | +- `runAt` / `notBefore` / `finishedAt` / `processedAt` / `failedAt` - Timestamps |
| 67 | + |
| 68 | +Unique constraint on `[key, runAt]` ensures scheduled jobs don't duplicate. |
| 69 | + |
| 70 | +### Job Processing Flow |
| 71 | + |
| 72 | +1. **Enqueue**: Jobs are inserted into the database via `enqueue()` or `schedule()` |
| 73 | +2. **Poll**: Queue continuously polls for available jobs based on `pollInterval` |
| 74 | +3. **Dequeue**: Uses a raw SQL query with `FOR UPDATE SKIP LOCKED` to atomically claim a job |
| 75 | +4. **Execute**: Calls the user-provided worker function with the job and Prisma client |
| 76 | +5. **Completion**: Updates job with result/error, handles retries with exponential backoff |
| 77 | +6. **Cron**: If job has `cron` + `key`, schedules next occurrence automatically |
| 78 | + |
| 79 | +### Utilities (`src/utils/`) |
| 80 | + |
| 81 | +- `debug.ts` - Debug logging with `debug` package |
| 82 | +- `error.ts` - Error serialization for JSON storage |
| 83 | +- `prisma.ts` - Prisma helper utilities (table name resolution, escaping) |
| 84 | +- `string.ts` - String manipulation (uncapitalize, escape) |
| 85 | +- `time.ts` - Time utilities (delay calculation, timezone handling) |
| 86 | +- `stringify.ts` - BigInt-safe JSON serialization (`prepareForJson`, `restoreFromJson`) |
| 87 | + |
| 88 | +### Entry Point |
| 89 | + |
| 90 | +`src/index.ts` exports: |
| 91 | +- `createQueue<T, U>(options, worker)` - Factory function for creating queues |
| 92 | +- All types from `types.ts` |
| 93 | +- `PrismaQueue` and `PrismaJob` classes |
| 94 | +- Utility functions `prepareForJson` and `restoreFromJson` |
| 95 | + |
| 96 | +## Testing |
| 97 | + |
| 98 | +Tests are located in: |
| 99 | +- `src/PrismaQueue.spec.ts` - Main queue tests |
| 100 | +- `src/index.spec.ts` - Integration tests |
| 101 | +- `test/setup.ts` - Test setup and teardown |
| 102 | +- `test/utils/` - Testing utilities (client, queue helpers, debug, timing) |
| 103 | + |
| 104 | +Tests require a PostgreSQL database specified via `DATABASE_URL` environment variable. |
| 105 | + |
| 106 | +## Known Issues / TODOs |
| 107 | + |
| 108 | +The following improvements have been identified and should be addressed: |
| 109 | + |
| 110 | +1. **Transaction consistency in `enqueue()` method** (src/PrismaQueue.ts:188-209) |
| 111 | + - Issue: When `key && runAt` is true, `this.model.deleteMany()` is called instead of using the transactional `client` |
| 112 | + - Impact: Mixes transaction and non-transaction contexts, could lead to race conditions |
| 113 | + - Fix: Use `client[queueJobKey].deleteMany()` instead of `this.model.deleteMany()` |
| 114 | + |
| 115 | +2. **Worker should use transactional client** (src/PrismaQueue.ts:335) |
| 116 | + - Issue: Worker is called with `this.#prisma` instead of the transactional `client` |
| 117 | + - Impact: Worker's database operations aren't part of the same transaction as job updates |
| 118 | + - Fix: Pass `client` to the worker instead of `this.#prisma` |
| 119 | + - Note: This is a breaking change that requires updating the `JobWorker` type signature |
| 120 | + |
| 121 | +3. **`runAt` condition may miss jobs** (src/PrismaQueue.ts:334) |
| 122 | + - Issue: SQL query uses `runAt < NOW()` (strict less than) |
| 123 | + - Impact: Jobs scheduled for exactly the current second may be missed |
| 124 | + - Fix: Change to `runAt <= NOW()` to include jobs scheduled for the current second |
0 commit comments