Skip to content

Commit 3d278c1

Browse files
committed
Dataflow: documentation.
1 parent 92a5e9e commit 3d278c1

File tree

2 files changed

+44
-0
lines changed

2 files changed

+44
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ The design goals of this gem are:
3333
* Actor variant [Channel](https://github.com/jdantonio/concurrent-ruby/blob/master/md/channel.md)
3434
loosely based on the [MailboxProcessor](http://blogs.msdn.com/b/dsyme/archive/2010/02/15/async-and-parallel-design-patterns-in-f-part-3-agents.aspx)
3535
agent in [F#](http://msdn.microsoft.com/en-us/library/ee370357.aspx)
36+
* [Dataflow](https://github.com/jdantonio/concurrent-ruby/blob/master/md/dataflow.md) loosely based on the syntax of Akka and Habanero Java
3637

3738
### Semantic Versioning
3839

@@ -111,6 +112,7 @@ These tools will help ease the burden, but at the end of the day it is essential
111112
* [Chip Miller](https://github.com/chip-miller)
112113
* [Jamie Hodge](https://github.com/jamiehodge)
113114
* [Zander Hill](https://github.com/zph)
115+
* [Chris Seaton](https://github.com/chrisseaton)
114116

115117
## Contributing
116118

md/dataflow.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,49 @@
11
# Dataflow
22

3+
Dataflow allows you to create a task that will be scheduled then all of its data
4+
dependencies are available. Data dependencies are `Future` values. The dataflow
5+
task itself is also a `Future` value, so you can build up a graph of these
6+
tasks, each of which is run when all the data and other tasks it depends on are
7+
available or completed.
8+
9+
Our syntax is somewhat related to that of Akka's `flow` and Habanero Java's
10+
`DataDrivenFuture`. However unlike Akka we don't schedule a task at all until it
11+
is ready to run, and unlike Habanero Java we pass the data values into the task
12+
instead of dereferencing them again in the task.
13+
14+
The theory of dataflow goes back to the 80s. In the terminology of the
15+
literature, our implementation is coarse-grained, in that each task can be many
16+
instructions, and dynamic in that you can create more tasks within other tasks.
17+
18+
## Example
19+
20+
A dataflow task is created with the `dataflow` method, passing in a block.
21+
22+
```ruby
23+
task = Concurrent::dataflow { 14 }
24+
```
25+
26+
This produces a simple `Future` value. The task will run immediately, as it has
27+
no dependencies. We can also specify `Future` values that must be available
28+
before a task will run. When we do this we get the value of those futures passed
29+
to our block.
30+
31+
```ruby
32+
a = Concurrent::dataflow { 1 }
33+
b = Concurrent::dataflow { 2 }
34+
c = Concurrent::dataflow(a, b) { |av, bv| av + bv }
35+
```
36+
37+
Using the `dataflow` method you can build up a directed acyclic graph (DAG) of
38+
tasks that depend on each other, and have the tasks run as soon as their
39+
dependencies are ready and there is CPU capacity to schedule them. This can help
40+
you create a program that uses more of the CPU resources available to you.
41+
342
## Derivation
443

44+
This section describes how we could derive dataflow from other primitives in
45+
this library.
46+
547
Consider a naive fibonacci calculator.
648

749
```ruby

0 commit comments

Comments
 (0)