|
1 | 1 | # JobGuard |
2 | 2 |
|
3 | | -[](https://www.npmjs.com/package/jobguard) |
4 | | -[](https://opensource.org/licenses/MIT) |
5 | | -[](https://www.typescriptlang.org/) |
6 | | -[](https://nodejs.org/) |
| 3 | +[](https://www.npmjs.com/package/jobguard) |
| 4 | +[](https://nodejs.org/) |
| 5 | +[](https://www.typescriptlang.org/) |
| 6 | +[](https://github.com/alexpota/jobguard/actions) |
| 7 | +[](https://github.com/alexpota/jobguard) |
| 8 | +[](https://opensource.org/licenses/MIT) |
| 9 | +[](https://www.npmjs.com/package/jobguard) |
7 | 10 |
|
8 | 11 | PostgreSQL durability for Redis-backed job queues (Bull, BullMQ, Bee-Queue) with minimal integration. |
9 | 12 |
|
| 13 | +## Quick Start |
| 14 | + |
| 15 | +### Installation |
| 16 | + |
| 17 | +```bash |
| 18 | +npm install jobguard pg |
| 19 | +``` |
| 20 | + |
| 21 | +### Basic Usage |
| 22 | + |
| 23 | +```typescript |
| 24 | +import Bull from 'bull'; |
| 25 | +import { JobGuard } from 'jobguard'; |
| 26 | + |
| 27 | +// Create your queue as usual |
| 28 | +const queue = new Bull('my-queue', 'redis://localhost:6379'); |
| 29 | + |
| 30 | +// Add JobGuard for durability |
| 31 | +const jobGuard = await JobGuard.create(queue, { |
| 32 | + postgres: 'postgresql://localhost:5432/mydb', |
| 33 | +}); |
| 34 | + |
| 35 | +// Use your queue normally - JobGuard works transparently |
| 36 | +await queue.add('email', { to: 'user@example.com' }); |
| 37 | + |
| 38 | +// Gracefully shutdown when done |
| 39 | +process.on('SIGTERM', async () => { |
| 40 | + await jobGuard.shutdown(); |
| 41 | + await queue.close(); |
| 42 | +}); |
| 43 | +``` |
| 44 | + |
| 45 | +## 🎬 Demo |
| 46 | + |
| 47 | + |
| 48 | + |
| 49 | +✅ **10,000 jobs • 60 workers • Redis crash at peak load • Zero jobs lost** |
| 50 | + |
| 51 | +[▶️ Run the interactive demo yourself →](./demo#readme) |
| 52 | + |
| 53 | +## Features |
| 54 | + |
| 55 | +- 🔒 **Drop-In Integration**: Wraps existing queues without modifying your queue code |
| 56 | +- 🔄 **Automatic Recovery**: Client-side reconciliation detects and recovers stuck jobs |
| 57 | +- 💓 **Heartbeat Support**: Long-running jobs signal liveness for accurate stuck detection |
| 58 | +- 📊 **Multi-Queue Support**: Works with Bull, BullMQ, and Bee-Queue |
| 59 | +- ⚡ **Low Overhead**: <5ms per job operation, minimal memory footprint |
| 60 | +- 🛡️ **Fault Tolerant**: Circuit breaker pattern protects against PostgreSQL failures |
| 61 | +- 🎯 **Type Safe**: Full TypeScript support with strict typing |
| 62 | + |
10 | 63 | ## Table of Contents |
11 | 64 |
|
12 | | -- [Why JobGuard?](#why-jobguard) |
| 65 | +- [Quick Start](#quick-start) |
13 | 66 | - [Demo](#-demo) |
14 | 67 | - [Features](#features) |
15 | | -- [Installation](#installation) |
16 | | -- [Quick Start](#quick-start) |
17 | | -- [Configuration](#configuration) |
| 68 | +- [Why JobGuard?](#why-jobguard) |
18 | 69 | - [Database Setup](#database-setup) |
| 70 | +- [Configuration](#configuration) |
19 | 71 | - [Advanced Usage](#advanced-usage) |
20 | 72 | - [API Reference](#api-reference) |
21 | 73 | - [Queue Library Support](#queue-library-support) |
@@ -76,63 +128,34 @@ const jobGuard = await JobGuard.create(queue, { |
76 | 128 |
|
77 | 129 | [▶️ Run the interactive stress test yourself](./demo#readme) |
78 | 130 |
|
79 | | -### Choose JobGuard If You: |
80 | | - |
81 | | -- ✅ Already use Bull/BullMQ/Bee-Queue and want to keep Redis performance |
82 | | -- ✅ Need 100% durability without rewriting existing queue code |
83 | | -- ✅ Want automatic recovery from Redis crashes |
84 | | -- ✅ Need to support multiple queue libraries in one codebase |
85 | | -- ✅ Want minimal integration effort (3 lines of code) |
86 | | - |
87 | | -## 🎬 Demo |
88 | | - |
89 | | - |
90 | | - |
91 | | -✅ **10,000 jobs • 60 workers • Redis crash at peak load • Zero jobs lost** |
92 | | - |
93 | | -[▶️ Run the interactive demo yourself →](./demo#readme) |
94 | | - |
95 | | -## Features |
96 | | - |
97 | | -- 🔒 **Drop-In Integration**: Wraps existing queues without modifying your queue code |
98 | | -- 🔄 **Automatic Recovery**: Client-side reconciliation detects and recovers stuck jobs |
99 | | -- 💓 **Heartbeat Support**: Long-running jobs signal liveness for accurate stuck detection |
100 | | -- 📊 **Multi-Queue Support**: Works with Bull, BullMQ, and Bee-Queue |
101 | | -- ⚡ **Low Overhead**: <5ms per job operation, minimal memory footprint |
102 | | -- 🛡️ **Fault Tolerant**: Circuit breaker pattern protects against PostgreSQL failures |
103 | | -- 🎯 **Type Safe**: Full TypeScript support with strict typing |
| 131 | +## Database Setup |
104 | 132 |
|
105 | | -## Quick Start |
| 133 | +**One-time setup:** Create the JobGuard table in your PostgreSQL database. |
106 | 134 |
|
107 | | -### Installation |
| 135 | +### Option 1: Using psql (Recommended) |
108 | 136 |
|
109 | 137 | ```bash |
110 | | -npm install jobguard pg |
| 138 | +psql -d mydb -f node_modules/jobguard/schema/001_initial.sql |
111 | 139 | ``` |
112 | 140 |
|
113 | | -### Basic Usage |
| 141 | +### Option 2: Programmatically |
114 | 142 |
|
115 | 143 | ```typescript |
116 | | -import Bull from 'bull'; |
117 | | -import { JobGuard } from 'jobguard'; |
118 | | - |
119 | | -// Create your queue as usual |
120 | | -const queue = new Bull('my-queue', 'redis://localhost:6379'); |
| 144 | +import { Pool } from 'pg'; |
| 145 | +import { readFileSync } from 'fs'; |
| 146 | +import { join } from 'path'; |
121 | 147 |
|
122 | | -// Add JobGuard for durability |
123 | | -const jobGuard = await JobGuard.create(queue, { |
124 | | - postgres: 'postgresql://localhost:5432/mydb', |
125 | | -}); |
| 148 | +const pool = new Pool({ connectionString: 'postgresql://localhost:5432/mydb' }); |
| 149 | +const schema = readFileSync( |
| 150 | + join(__dirname, 'node_modules/jobguard/schema/001_initial.sql'), |
| 151 | + 'utf8' |
| 152 | +); |
| 153 | +await pool.query(schema); |
| 154 | +``` |
126 | 155 |
|
127 | | -// Use your queue normally - JobGuard works transparently |
128 | | -await queue.add('email', { to: 'user@example.com' }); |
| 156 | +### Option 3: Add to Your Existing Migrations |
129 | 157 |
|
130 | | -// Gracefully shutdown when done |
131 | | -process.on('SIGTERM', async () => { |
132 | | - await jobGuard.shutdown(); |
133 | | - await queue.close(); |
134 | | -}); |
135 | | -``` |
| 158 | +Copy `node_modules/jobguard/schema/001_initial.sql` into your project's migration system (Knex, TypeORM, Prisma, etc.). |
136 | 159 |
|
137 | 160 | ## Configuration |
138 | 161 |
|
@@ -181,35 +204,6 @@ const jobGuard = await JobGuard.create(queue, { |
181 | 204 | }); |
182 | 205 | ``` |
183 | 206 |
|
184 | | -## Database Setup |
185 | | - |
186 | | -**One-time setup:** Create the JobGuard table in your PostgreSQL database. |
187 | | - |
188 | | -### Option 1: Using psql (Recommended) |
189 | | - |
190 | | -```bash |
191 | | -psql -d mydb -f node_modules/jobguard/schema/001_initial.sql |
192 | | -``` |
193 | | - |
194 | | -### Option 2: Programmatically |
195 | | - |
196 | | -```typescript |
197 | | -import { Pool } from 'pg'; |
198 | | -import { readFileSync } from 'fs'; |
199 | | -import { join } from 'path'; |
200 | | - |
201 | | -const pool = new Pool({ connectionString: 'postgresql://localhost:5432/mydb' }); |
202 | | -const schema = readFileSync( |
203 | | - join(__dirname, 'node_modules/jobguard/schema/001_initial.sql'), |
204 | | - 'utf8' |
205 | | -); |
206 | | -await pool.query(schema); |
207 | | -``` |
208 | | - |
209 | | -### Option 3: Add to Your Existing Migrations |
210 | | - |
211 | | -Copy `node_modules/jobguard/schema/001_initial.sql` into your project's migration system (Knex, TypeORM, Prisma, etc.). |
212 | | - |
213 | 207 | ## Advanced Usage |
214 | 208 |
|
215 | 209 | ### Force Reconciliation |
|
0 commit comments