CadenceMQ - Job scheduling library

CadenceMQ is a job scheduling library for Node.js. It is designed to be easy to use, flexible, and scalable.

Note

This is a work in progress, anything can change at any time.

Features

• Job scheduling
• Retry mechanism
• Periodic jobs
• Job status tracking
• UI for monitoring and managing jobs
• Unique key for jobs
• Drivers for different backends
  • SQLite/LibSQL (in-memory, remote server or file-based)
  • Native in-memory
  • Redis
  • MongoDB

Why another job scheduling library?

There are many job scheduling libraries of quality, but most of them are tied to a specific backend, like BullMQ with Redis or Agenda with MongoDB. CadenceMQ is designed to be backend agnostic, and can be used with different backends, the motivation is to provide a simple simple for self-hostable applications but being able to scale horizontally in production environments.

This as been initially created for Papra, a self-hostable minimalistic document management platform, as we didn't want the self-hosters to have to setup a Redis.

Getting started

Basic example using LibSQL/SQLite as backend:

# using pnpm
pnpm install @cadence-mq/core @cadence-mq/driver-libsql

# using bun
bun add @cadence-mq/core @cadence-mq/driver-libsql

# using npm
npm install @cadence-mq/core @cadence-mq/driver-libsql

# using yarn
yarn add @cadence-mq/core @cadence-mq/driver-libsql

import { createCadence } from '@cadence-mq/core';
import { createLibSqlDriver } from '@cadence-mq/driver-libsql';
import { createClient } from '@libsql/client';
import { z } from 'zod';

const driver = createLibSqlDriver({ client: createClient({ url: ':memory:' }) });

const cadence = createCadence({ driver });

cadence.registerTask({
  taskName: 'send-welcome-email',
  handler: async ({ data }) => {
    console.log('Sending welcome email to', (data as { email: string }).email);
  },
});

const worker = cadence.createWorker({ workerId: '1' });

worker.start();

await cadence.scheduleJob({
  taskName: 'send-welcome-email',
  data: { email: 'test@test.com' },
});

Examples

• Local SQLite

Contributing

Contributions are welcome! Please refer to the CONTRIBUTING.md file for guidelines on how to get started, report issues, and submit pull requests.

Credits

This project is crafted with ❤️ by Corentin Thomasset. If you find this project helpful, please consider supporting my work.

Acknowledgements

CadenceMQ is inspired by some great projects:

• Agenda, a task runner and scheduler based on MongoDB.
• Pulse, a fork of Agenda with extended features.
• BullMQ, a message queue and batch processing for NodeJS and Python based on Redis.
• Plainjob, a SQLite-backed job queue.
• A SQLite Background Job System, an article by Jason Gorman.