Dependencies between tasks

[kirillsalykin]

hi,

Huge thanks for the underway!
I've had implemented something similar to the underway, but in clojure.

We also needed a deps between jobs, for instance - before the email can be sent - pdf document should be generated; maybe this is something useful here as well?

it was pretty straightforward to implement, not complex DAG; based on table like this:

sql

CREATE TABLE jobs_deps (
  id SERIAL PRIMARY KEY,
  dependent_id INT NOT NULL REFERENCES jobs(id),
  dependency_id INT NOT NULL REFERENCES jobs(id),
  created_at timestamp with time zone NOT NULL DEFAULT NOW()
);

and it also changes how you pull next job - you want to ensure that there are no not completed dependend jobs.

probably it also requires to have something like `enqueue_with_deps(jobs: Vec).

WDYT?

[maxcountryman]

I've been working on an extension of the jobs abstraction that has exactly this in mind.

If you have time to take a look, I'd love your thoughts about the design in the step functions branch.

[maxcountryman]

Thanks for that! I need to update the documentation and then should be able to cut a new 0.0.x release from the step functions branch. As you're testing, I'd love to read feedback you might have. 🙂

[maxcountryman]

@kirillsalykin I just published 0.0.5. I also spent some time updating the documentation. In particular, if you have feedback about the job module level docs, that would be helpful. Apart from that, if you run into any problems please do let me know. It's also worth mentioning that there's missing API still (unschedule, task cancellation, etc).

[kirillsalykin]

Will do, thanks

[kirillsalykin]

Tasks represent a well-structure unit of work.

I think there is a small typo, well structure**d** should it be?

[kirillsalykin]

the docs are looking awesome, clean and clear!

Also, I must admit step's are pretty cool.