Skip to content

Commit a262204

Browse files
committed
Merge pull request #4 from dabapps/docs-update
Update readme
2 parents d3e7ea9 + b51ff85 commit a262204

File tree

1 file changed

+53
-7
lines changed

1 file changed

+53
-7
lines changed

README.md

Lines changed: 53 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,68 @@
11
django-db-queue
22
==========
33
[![Build Status](https://travis-ci.org/dabapps/django-db-queue.svg)](https://travis-ci.org/dabapps/django-db-queue)
4+
[![pypi release](https://img.shields.io/pypi/v/django-db-queue.svg)](https://pypi.python.org/pypi/django-db-queue)
45

56
Simple databased-backed job queue. Jobs are defined in your settings, and are processed by management commands.
67

78
Asynchronous tasks are run via a *job queue*. This system is designed to support multi-step job workflows.
89

910
This is not yet production-ready, and the API is liable to change. Use at your own risk.
1011

11-
### Terminology
12+
## Getting Started
1213

13-
#### Job
14+
### Describe your job
15+
16+
In e.g. project.common.jobs:
17+
18+
```python
19+
import time
20+
21+
def my_task(job):
22+
logger.info("Working hard...")
23+
time.sleep(10)
24+
logger.info("Job's done!")
25+
```
26+
27+
### Set up your job
28+
29+
In project.settings:
30+
31+
```python
32+
JOBS = {
33+
'my_job': ['project.common.jobs.my_task'],
34+
}
35+
```
36+
37+
### Start the worker
38+
39+
In another terminal:
40+
41+
`python manage.py worker`
42+
43+
### Create a job
44+
45+
Using the name you configured for your job in your settings, create an instance of Job.
46+
47+
```python
48+
Job.objects.create(name='my_job')
49+
```
50+
51+
## Terminology
52+
53+
### Job
1454

1555
The top-level abstraction of a standalone piece of work. Jobs are stored in the database (ie they are represented as Django model instances).
1656

17-
#### Task
57+
### Task
1858

1959
Jobs are processed to completion by *tasks*. These are simply Python functions, which must take a single argument - the `Job` instance being processed. A single job will often require processing by more than one task to be completed fully. Creating the task functions is the responsibility of the developer. For example:
2060

2161
def my_task(job):
2262
logger.info("Doing some hard work")
2363
do_some_hard_work()
2464

25-
#### Workspace
65+
### Workspace
2666

2767
The *workspace* is an area that tasks within a single job can use to communicate with each other. It is implemented as a Python dictionary, available on the `job` instance passed to tasks as `job.workspace`. The initial workspace of a job can be empty, or can contain some parameters that the tasks require (for example, API access tokens, account IDs etc). A single task can edit the workspace, and the modified workspace will be passed on to the next task in the sequence. For example:
2868

@@ -32,16 +72,22 @@ The *workspace* is an area that tasks within a single job can use to communicate
3272
def my_second_task(job):
3373
logger.info("Task 1 says: %s" % job.workspace['message'])
3474

35-
#### Worker process
75+
When creating a Job, the workspace is passed as a keyword argument:
76+
77+
```python
78+
Job.objects.create(name='my_job', workspace={'key': value})
79+
```
80+
81+
### Worker process
3682

3783
A *worker process* is a long-running process, implemented as a Django management command, which is responsible for executing the tasks associated with a job. There may be many worker processes running concurrently in the final system. Worker processes wait for a new job to be created in the database, and call the each associated task in the correct sequeunce.. A worker can be started using `python manage.py worker`, and a single worker instance is included in the development `procfile`.
3884

3985
### Configuration
4086

41-
Jobs are configured in the Django `settings.py` file. The `JOBS` setting is a dictionary mapping a *job name* (eg `import_hats`) to a *list* of one or more task function paths. For example:
87+
Jobs are configured in the Django `settings.py` file. The `JOBS` setting is a dictionary mapping a *job name* (eg `import_cats`) to a *list* of one or more task function paths. For example:
4288

4389
JOBS = {
44-
'import_hats': ['apps.hat_hatter.import_hats.step_one', 'apps.hat_hatter.import_hats.step_two'],
90+
'import_cats': ['apps.cat_importer.import_cats.step_one', 'apps.cat_importer.import_cats.step_two'],
4591
}
4692

4793
### Job states

0 commit comments

Comments
 (0)