Support cljdoc

Update documentation and metadata

Author: KingMob

.gitignore

@@ -8,6 +8,4 @@ pom.xml.asc
 /.lein-*
 /.nrepl-port
 *DS_Store
-/doc
 push
-/doc

README.md

@@ -1,13 +1,13 @@
 [![Clojars Project](https://img.shields.io/clojars/v/manifold.svg)](https://clojars.org/manifold)
 [![cljdoc badge](https://cljdoc.org/badge/manifold/manifold)](https://cljdoc.org/d/manifold/manifold)
 [![CircleCI](https://circleci.com/gh/clj-commons/manifold.svg?style=svg)](https://circleci.com/gh/clj-commons/manifold)
-![](docs/manifold.png)
+![](doc/manifold.png)
 
-This library provides basic building blocks for asynchronous programming, and can be used as a translation layer between libraries which use similar but incompatible abstractions.
+Manifold provides basic building blocks for asynchronous programming, and can be used as a translation layer between libraries which use similar, but incompatible, abstractions.
 
 Manifold provides two core abstractions: **deferreds**, which represent a single asynchronous value, and **streams**, which represent an ordered sequence of asynchronous values.
 
-A detailed discussion of Manifold's rationale can be found [here](http://aleph.io/manifold/rationale.html).  Full documentation can be found [here](https://cljdoc.org/d/manifold/manifold).
+A detailed discussion of Manifold's rationale can be found [here](doc/rationale.md).  Full documentation can be found [here](https://cljdoc.org/d/manifold/manifold).
 
 
 ```clojure
@@ -59,7 +59,7 @@ success! :foo
 true
 ```
 
-Callbacks are a useful building block, but they're a painful way to create asynchronous workflows.  In practice, **no one should ever need to use `on-realized`**.  Manifold provides a number of operators for composing over deferred values, [which can be read about here](/docs/deferred.md).
+Callbacks are a useful building block, but they're a painful way to create asynchronous workflows.  In practice, **no one should ever need to use `on-realized`**.  Manifold provides a number of operators for composing over deferred values, [which can be read about here](/doc/deferred.md).
 
 ### Streams
 
@@ -110,7 +110,7 @@ nil
 1
 ```
 
-Manifold can use any transducer, which are applied via `transform`.  It also provides stream-specific transforms, including `zip`, `reduce`, `buffer`, `batch`, and `throttle`.  [To learn more about streams, go here](/docs/stream.md).
+Manifold can use any transducer, which are applied via `transform`.  It also provides stream-specific transforms, including `zip`, `reduce`, `buffer`, `batch`, and `throttle`.  [To learn more about streams, go here](/doc/stream.md).
 
 ### Clojurescript
 

doc/cljdoc.edn

@@ -0,0 +1,5 @@
+{:cljdoc.doc/tree [["Read Me" {:file "README.md"}]
+                   ["Rationale" {:file "doc/rationale.md"}]
+                   ["Deferreds" {:file "doc/deferred.md"}]
+                   ["Streams" {:file "doc/stream.md"}]
+                   ["Execution" {:file "doc/execution.md"}]]}

docs/deferred.md renamed to doc/deferred.md

@@ -131,8 +131,8 @@ Let's say that we have two services which provide us numbers, and want to get th
   (let [a (call-service-a)
         b (call-service-b)]
     (chain (zip a b)
-      (fn [[a b]]
-        (+ a b)))))
+           (fn [[a b]]
+             (+ a b)))))
 ```
 
 However, this isn't a very direct expression of what we're doing.  For more complex relationships between deferred values, our code will become even more difficult to understand.  In these cases, it's often best to use `let-flow`.
@@ -158,38 +158,45 @@ but not this:
 ```clojure
 (let-flow [a (future 1)
            b (let [c (future 1)]
-                (+ a c))]
+               (+ a c))]
   (+ b 1))
 ```
 
 In this example, `c` is declared within a normal `let` binding, and as such we can't treat it as if it were realized.
 
 It can be helpful to think of `let-flow` as similar to Prismatic's [Graph](https://github.com/prismatic/plumbing#graph-the-functional-swiss-army-knife) library, except that the dependencies between values are inferred from the code, rather than explicitly specified.  Comparisons to core.async's goroutines are less accurate, since `let-flow` allows for concurrent execution of independent paths within the bindings, whereas operations within a goroutine are inherently sequential.
 
-### manifold.go-off
+### go-off
 
-An alternate way to write code using deferreds is the macro `manifold.go-off/go-off`. This macro is an almost exact mirror of the `go` macro from [clojure/core.async](https://github.com/clojure/core.async), to the point where it actually utilizes the state machine functionality from core.async. In order to use this macro, `core.async` must be a dependency provided by the user. The main difference between `go` and `go-off`, besides go-off working with deferrables instead of core.async channels, is the `take` function being `<!?` instead of `<!`. The difference in function names is used to indicate exceptions behave the same as a non-async clojure block (i.e. are thrown) instead of silently swallowed & returning `nil`.  
+An alternate way to write code using deferreds is the macro `manifold.go-off/go-off`. This macro is an almost-exact mirror of the `go` macro from [core.async](https://github.com/clojure/core.async), to the point where it actually utilizes the state machine functionality from core.async. In order to use this macro, `core.async` must be provided as a dependency by the user. 
 
-The benefit of this macro over `let-flow` is that it gives complete control of when deferreds should be realized to the user of the macro, removing any potential surprises (especially around timeouts).
+There are a few major differences between `go` and `go-off`. First, `go-off` (unsurprisingly) works with Manifold deferreds and streams instead of core.async channels. Second, in addition to the `<!` fn, which always returns a value, there's also the `<!?` macro, which behaves the same unless a Throwable is retrieved, in which case it's automatically rethrown for you. Finally, there's no `>!` equivalent in `go-off`, as there's no way without altering the syntax to distinguish between success and error when putting into a deferred.
 
-```clj
+The benefit of `go-off` over `let-flow` is that it gives complete control of when deferreds should be realized to the user, removing any potential surprises (especially around timeouts).
+
+```clojure
+;; basic usage
 @(go-off (+ (<!? (d/future 10))
-             (<!? (d/future 20))))                          ;; ==> 30
+            (<!? (d/future 20))))       ;; 30
 ```
 
-```clj
-(<!! (core.async/go (try (<! (go (/ 5 0)))
-                         (catch Exception e
-                           "ERROR"))))                      ; ==> nil
+```clojure
+;; <!? usage
+
+;; if you forget to catch an exception in go, it won't be returned
+(<!! (go (try (<! (go (/ 5 0)))        
+              (catch Exception e
+                "ERROR"))))             ;; nil
 
+;; d/future returns any exceptions, and <!? rethrows it for you 
 @(go-off (try (<!? (d/future (/ 5 0)))
-               (catch Exception e
-                 "ERROR")))                                 ; ==> "ERROR"
+              (catch Exception e
+                "ERROR")))              ;; "ERROR"
 ```
 
-### `manifold.deferred/loop`
+### loop
 
-Manifold also provides a `loop` macro, which allows for asynchronous loops to be defined.  Consider `manifold.stream/consume`, which allows a function to be invoked with each new message from a stream.  We can implement similar behavior like so:
+Manifold also provides a `loop` macro, which allows for asynchronous loops to be defined. Consider `manifold.stream/consume`, which allows a function to be invoked with each new message from a stream.  We can implement similar behavior like so:
 
 ```clojure
 (require
@@ -215,8 +222,7 @@ Manifold also provides a `loop` macro, which allows for asynchronous loops to be
 
 Here we define a loop which takes messages one at a time from `stream`, and passes them into `f`.  If `f` returns an unrealized value, the loop will pause until it's realized.  To recur, we make sure the value returned from the final stage is `(manifold.deferred/recur & args)`, which will cause the loop to begin again from the top.
 
-While Manifold doesn't provide anything as general purpose as core.async's `go` macro, the combination of `loop` and `let-flow` can allow for the specification of highly intricate asynchronous workflows.
 
 ### Custom execution models
 
-Both deferreds and streams allow for custom execution models to be specified.  To learn more, [go here](/docs/execution.md).
+Both deferreds and streams allow for custom execution models to be specified.  To learn more, [go here](/doc/execution.md).

docs/execution.md renamed to doc/execution.md

@@ -4,7 +4,7 @@ This means that we not only need to correctly process the data, we need to have
 
 Backpressure is a fundamental property of Java's threading model, as shown by `BlockingQueues`, `Futures`, `InputStreams`, and others.  It's also a fundamental property of `core.async` channels, though it uses a completely different execution model built on callbacks and macros.  It's also a fundamental property of Clojure's lazy sequences, which like Java's abstractions are blocking, but unlike both Java and `core.async` relies on pulling data towards the consumer rather than having it pushed.
 
-Unfortunately, while all of these abstractions (or [RxJava](https://github.com/Netflix/RxJava), or [Reactive Streams](http://www.reactive-streams.org/), or ...) can be used to similar effects, they don't necessarily work well with each other.  The practical effect of this is that by choosing one abstraction, we often make the others off-limits.  When writing an application, this may be acceptable, if not really desirable.  When writing a library or something meant to be reused, though, it's much worse; only people who have chosen your particular walled garden can use your work.
+Unfortunately, while all of these abstractions (or [RxJava](https://github.com/ReactiveX/RxJava), or [Reactive Streams](http://www.reactive-streams.org/), or ...) can be used to similar effects, they don't necessarily work well with each other.  The practical effect of this is that by choosing one abstraction, we often make the others off-limits.  When writing an application, this may be acceptable, if not really desirable.  When writing a library or something meant to be reused, though, it's much worse; only people who have chosen your particular walled garden can use your work.
 
 Manifold provides abstractions that sits at the intersection of all these similar, but incompatible, approaches.  It provides an extensible mechanism for coercing unrealized data into a generic form, and piping data from these generic forms into other stream representations.
 

docs/manifold.png renamed to doc/manifold.png

@@ -18,7 +18,7 @@ A stream can be thought of as two separate halves: a **sink**  which consumes me
 << 1 >>
 ```
 
-Notice that both `put!` and `take!` return [deferred values](/docs/deferred.md).  The deferred returned by `put!` will yield `true` if the message was accepted by the stream, and `false` otherwise; the deferred returned by `take!` will yield the message.
+Notice that both `put!` and `take!` return [deferred values](/doc/deferred.md).  The deferred returned by `put!` will yield `true` if the message was accepted by the stream, and `false` otherwise; the deferred returned by `take!` will yield the message.
 
 Sinks can be **closed** by calling `close!`, which means they will no longer accept messages.
 

docs/rationale.md renamed to doc/rationale.md

@@ -1,4 +1,5 @@
-(ns manifold.debug)
+(ns manifold.debug
+  {:no-doc true})
 
 (def ^:dynamic *dropped-error-logging-enabled?* true)
 

docs/stream.md renamed to doc/stream.md

@@ -1,6 +1,7 @@
-(ns ^{:author "Ryan Smith"
-      :doc    "Provide a variant of `core.async/go` that works with manifold's deferreds and executors. Utilizes core.async's state-machine generator, so core.async must be provided as a dependency."}
-  manifold.go-off
+(ns manifold.go-off
+  {:author "Ryan Smith"
+   :doc    "Provides a variant of `core.async/go` that works with manifold's deferreds and streams. Utilizes core.async's state-machine generator, so core.async must be provided by consumers as a dependency."
+   :added "0.2.0"}
   (:require [manifold
              [executor :as ex]
              [deferred :as d]]
@@ -15,21 +16,21 @@
     d))
 
 (defn <!
-  "Takes value from a deferred/stream. Must be called inside a (go ...) block. Will
+  "Takes value from a deferred/stream. Must be called inside a `(go-off ...)` block. Will
    return nil if a stream is closed. Will park if nothing is available. If an error
-   is thrown inside the body, that error will be placed as the return value.
+   is thrown inside the body, that error will be the return value.
 
    N.B. To make `go-off` usage idiomatic with the rest of manifold, use `<!?`
    instead."
-  [port]
+  [deferred-or-stream]
   (assert nil "<! used not in (go-off ...) block"))
 
 (defmacro <!?
-  "Takes a val from a deferred/stream. Must be called inside a (go-off ...) block.
-   Will park if nothing is available. If value that is returned is a Throwable,
-   it will re-throw."
-  [port]
-  `(let [r# (<! ~port)]
+  "Takes a value from a deferred/stream. Must be called inside a `(go-off ...)` block.
+   Will park if nothing is available. If the value returned is a Throwable,
+   it will be re-thrown."
+  [deferred-or-stream]
+  `(let [r# (<! ~deferred-or-stream)]
      (if (instance? Throwable r#)
        ;; this is a re-throw of the original throwable. the expectation is that
        ;; it still will maintain the original stack trace
@@ -42,7 +43,7 @@
          (d/error! (ioc/aget-object state ioc/USER-START-IDX) ex)
          (throw ex))))
 
-(defn take! [state blk d]
+(defn ^:no-doc take! [state blk d]
   (let [handler          (fn [x]
                            (ioc/aset-all! state ioc/VALUE-IDX x ioc/STATE-IDX blk)
                            (run-state-machine-wrapped state))
@@ -72,7 +73,7 @@
    :Return             `return-deferred})
 
 (defmacro go-off-with
-  "Implementation of go-off that allows specifying executor. See docstring of go-off for usage."
+  "An implementation of `go-off` that allows specifying the executor to run on. See docstring of [[go-off]] for usage."
   [executor & body]
   (let [executor     (vary-meta executor assoc :tag 'java.util.concurrent.Executor)
         crossing-env (zipmap (keys &env) (repeatedly gensym))]
@@ -92,8 +93,8 @@
 
 (defmacro go-off
   "Asynchronously executes the body on manifold's default executor, returning
-   immediately to the calling thread. Additionally, any visible calls to <!?
-   and <! deferred operations within the body will block (if necessary)
+   immediately to the calling thread. Additionally, any visible calls to `<!?`
+   and `<!` deferred operations within the body will block (if necessary)
    by 'parking' the calling thread rather than tying up an OS thread.
    Upon completion of the operation, the body will be resumed.
 
@@ -106,16 +107,16 @@
    to address the following major points from core.async & vanilla manifold deferreds:
 
    - `core.async/go` assumes that all of your code is able to be purely async
-   and will never block the handling threads. go-off removes the concept of handling
-   threads, which means blocking is not an issue, but if you spawn too many of these you
-   can create too many threads for the OS to handle.
+   and will never block the handling threads. `go-off` removes the concept of handling
+   threads, which means blocking is not an issue, but if you spawn too many of these, you
+   can create more threads than the OS can handle.
    - `core.async/go` has no built-in way of handling exceptions and assumes all async
    code will be either written defensively, or have custom error propagation, which
    differs from how clojure code blocks typically work outside of the async world.
-   - `deferred/let-flow` presumes that every deferrable needs to be resolved. This prevents
+   - [[deferred/let-flow]] presumes that every deferrable needs to be resolved. This prevents
    more complex handling of parallelism or being able to pass deferreds into other functions
    from within the `let-flow` block.
-   - `deferred/chain` only works with single deferreds, which means having to write code in
+   - [[deferred/chain]] only works with single deferreds, which means having to write code in
    unnatural ways to handle multiple deferreds."
   [& body]
   `(go-off-with (ex/execute-pool) ~@body))