Monitoring CLI / UI & Job cancellation

[maelp]

Would be nice to have a small monitoring CLI / UI to see the running tasks and progress, be able to cancel some, etc
For inspiration (possible integration?), check https://github.com/earendil-works/absurd/ which has a nice UI

[noxify]

Hi,

Thanks for the request.

That's an awesome idea.

I think the following features should be "easy" to solve:

• Listing jobs and their current status (pending, running, etc. )
• retry a job (I would simply create a clone of the failed job)

A bit more challenging could be the "cancel job" feature.

Cancelling a running job is currently not possible - if you have an idea how to make it work, let me know :)

Cancel a pending job should be easy, since we just mark them a cancelled or just delete them from the queue.

Another challenge could be how to fetch the information since we need a database connection and maybe the adapter - same here, if you have an idea, let me know.

For the UI approach, would you expect a standalone version or just some snippets to integrate it into your existing application?

[noxify]

Would be nice to have a small monitoring CLI / UI

I spent some time to dig into the "absurd" implementation.

Providing a "bundled" version with an ORM of my choice ( or just plain sql via pg or postgres.js ) should work.
I just have to make sure it's configurable ( see #40 where I try to make schema and table name configurable ).

Have seen that "absurd" uses vite for the UI, which should work, but have to compare it with a bundled nextjs app.
For the CLI approach, maybe sst/opentui would be cool to have an interactive tui 🤔

---

be able to cancel some

What do you think about this approach ( note: used copilot to get an inspiration ):

• a new status cancelled
• a new field cancelledAt
• a new method cancelJob on queue level, which can be called to trigger the cancellation ( and sets the status + cancelledAt )
• a new event job:cancelled
• a new event job:cancel:{jobId}

To process the cancellation, we're extending the processJob and processBatchJobs and listen to the job:cancel:${jobId} event and stop the processing ( inside processJob we just have to extend the Promise.race.

For processBatchJobs, we have to modify the current implementation ( maybe, we just have to call processJob from processBatchJobs 🤔 ).

This implementation is similar to the current timeout implementation and it seems to be a non-breaking change.

[maelp]

Indeed this should work! I guess if you're using Promise/async there's even a kind of "cancel" signal in promises, don't know if this could be useful?

[noxify]

Yeah, the hole processing logic runs inside a Promise. ( Ref: https://github.com/noxify/vorsteh-queue/blob/main/packages/core/src/core/queue.ts#L632-L640 ).

We have already a "cancel like" logic with the timeout implementation.

implementation could be something like this:

const timeout = job.timeout ?? this.config.defaultJobOptions.timeout ?? 30000

const cancelPromise = new Promise((resolve) => {
  jobEvents.once(`job:cancel:${job.id}`, resolve)
})

const result =
  timeout === false
    ? await Promise.race([
      handler(createJobWrapper(job, this)),
      cancelPromise
    ])
    : await Promise.race([
        handler(createJobWrapper(job, this)),
        new Promise((_, reject) =>
          setTimeout(() => reject(new Error("Job timeout")), timeout),
        ),
        cancelPromise
      ])

I think we have to check if there was a cancel event and skip the following block:

await this.adapter.updateJobStatus(job.id, "completed", undefined, result)
this.emit("job:completed", { ...job, status: "completed", completedAt: new Date(), result })

[maelp]

Indeed this could work! I was even thinking of AbortController to abort the promise, but not sure if it gives the level of control we'd need https://leanylabs.com/blog/cancel-promise-abortcontroller/

[noxify]

created a new issue for the implementation and will check if the AbortController could work, too.
How important is the cancellation for you.

[maelp]

I don't think it's so important TBH, but I was just thinking that if we do an UI, it's something that's quite "natural"

could be useful if for instance we do durable workflows (integrating with Absurd) and then you naturally want to cancel and re-run some old workflows if you've updated the underlying code in an update, etc