Skip to content

Update durable-queue to use deps.edn and the latest dependency versions #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .clj-kondo/config.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{:lint-as {clj-commons.durable-queue/with-buffer clojure.core/let}}
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,8 @@ pom.xml.asc
/.nrepl-port
.DS_Store
/doc
push
push
*.iml
.idea/
.cpcache/
.clj-kondo/.cache
28 changes: 17 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,33 @@ This library implements a disk-backed task queue, allowing for queues that can s

### usage

Leiningen:

```clj
[factual/durable-queue "0.1.5"]
[factual/durable-queue "0.1.6"]
```

deps.edn
```clj
io.github.vedang/durable-queue {:git/sha "8dfc74193b5a78608cfe4fba821dab8b8b5644be"}
```
To interact with queues, first create a `queues` object by specifying a directory in the filesystem and an options map:

```clj
> (require '[durable-queue :refer :all])
> (require '[clj-commons.durable-queue :as dq])
nil
> (def q (queues "/tmp" {}))
> (def q (dq/queues "/tmp" {}))
#'q
```

This allows us to `put!` and `take!` tasks from named queues. `take!` is a blocking read, and will only return once a task is available or, if a timeout is defined (in milliseconds), once the timeout elapses:

```clj
> (take! q :foo 10 :timed-out!)
> (dq/take! q :foo 10 :timed-out!)
:timed-out!
> (put! q :foo "a task")
> (dq/put! q :foo "a task")
true
> (take! q :foo)
> (dq/take! q :foo)
< :in-progress | "a task" >
> (deref *1)
"a task"
Expand All @@ -39,20 +45,20 @@ Notice that the task has a value describing its progress, and a value describing
Calling `take!` removed the task from the queue, but just because we've taken the task doesn't mean we've completed the action associated with it. In order to make sure the task isn't retried on restart, we must mark it as `complete!`.

```clj
> (put! q :foo "another task")
> (dq/put! q :foo "another task")
true
> (take! q :foo)
> (dq/take! q :foo)
< :in-progress | "another task" >
> (complete! *1)
> (dq/complete! *1)
true
```

If our task fails and we want to re-enqueue it to be tried again, we can instead call `(retry! task)`. Tasks which are marked for retry are added to the end of the current queue.
If our task fails and we want to re-enqueue it to be tried again, we can instead call `(dq/retry! task)`. Tasks which are marked for retry are added to the end of the current queue.

To get a description of the current state of the queue, we can use `stats`, which returns a map of queue names onto various counts:

```clj
> (stats q)
> (dq/stats q)
{:enqueued 2,
:retried 0,
:completed 1,
Expand Down
25 changes: 25 additions & 0 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{:paths ["src" "resources"]

:deps
{com.taoensso/nippy {:mvn/version "3.2.0"}
org.clj-commons/byte-streams {:mvn/version "0.3.2"}
org.clj-commons/primitive-math {:mvn/version "1.0.1-rc1"}}

:aliases
{:test
{:extra-paths ["test"]
:extra-deps {criterium/criterium {:mvn/version "0.4.6"}}}
;;; Run tests with cognitect-labs/test-runner
;; clojure -X:test:congnitect :excludes '[:stress :benchmark]'
;; NOTE: :stress and :benchmark generate a lot of slab files and
;; need a huge amount of disk space.
:cognitect
{:extra-deps {io.github.cognitect-labs/test-runner
{:git/tag "v0.5.1" :git/sha "dfb30dd"}}
:main-opts ["-m" "cognitect.test-runner"]
:exec-fn cognitect.test-runner.api/test}}

:description "a in-process task-queue that is backed by disk."
:url "https://github.com/clj-commons/durable-queue"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}}
23 changes: 0 additions & 23 deletions project.clj

This file was deleted.

Loading