Complex Job

Description

This document is not finished!!!!

I will complete this document after the features discussed in #47 have been added.

If you have a need to process complex jobs rethinkdb-job-queue may do the job for you. To help you decide, this document will describe some of the different ways you can employ the queue. The document is broken down into the following complex job processing types:

• Fixed Multi-Part Job
• Variable Multi-Part Job
• FIFO Job
• FILO Job
• Incomplete Job

Fixed Multi-Part Job

Like many modules on the NPM registry, rethinkdb-job-queue is a building block that you use within your application. Its primary function is to reliably process jobs in a distributed application framework. It can be used on a single server very successfully though.

If you have a complex task that has multiple states and sub tasks, the best way to handle these tasks in your application is by building the task logic into your application. If you break the high level task down into individual steps or jobs you can then rely on rethinkdb-job-queue to process those lower level jobs for you.

Here is an example of the logic required to build an application that has a complex registration process. The registration process requires three sub-tasks to be performed to complete the workflow as follows:

• Send registration activation email.
• Register user with third party service provider.
• Send confirmation email.

With the above three tasks we can use rethinkdb-job-queue to manage each task by creating three distinct queues; ActivationEmail, SPRegistration, and ConfirmationEmail queues.

For the management of the higher level task you will need to build this logic into your application and use your applications database to save each state.

The application logic would go something like this;

1. User registers on your web application.
2. Your application saves the registration to your application database (not the job queue).
3. In the same code that you save the registration to your database, initiate a job within the ActivationEmail queue to send an account activation email. Use database transactions for reliability.
4. At the completion of the ActivationEmail queue job, update the registration state in your application database.
5. Wait for the user to click the activation link.
6. After the user activates the new account, update the registration state in your application database.
7. In the same code that you update the registration state, initiate a job within the SPRegistration queue to register the new account with the third party service. Again, use database transactions for reliability.
8. At the completion of the SPRegistration queue job, update the registration state in your application database.
9. In the same code that you update the registration state, initiate a job within the ConfirmationEmail queue to send a congratulations or confirmation email to the new user.
10. At the completion of the ConfirmationEmail queue job, update the registration state to completed in your application database.

Please see the Tutorial, Queue Constructor, Queue PubSub, Job Processing, and Queue.process documents for more detail.

FIFO Job

First In First Out

FILO Job

First In Last Out

Incomplete Job