Derived Values

Derived Values

Transform functions receive messages and apply them to the data model. What if we need to have a value in the data model that is based on values which are modified by more than one transform function?

Derive functions allow us to compute new values from any other values in the data model. A derive function will be called when any of its inputs change. Derive functions can be arranged into an arbitrary dataflow.

In this step we will calculate some additional information based on the counters that we receive. While doing this we will be introduced to the following topics:

• Derive functions
• Dataflow
• Working with dataflow inputs

Remember to have the dev tools running before you start.

Deriving a total

One interesting value to have would be the total of all the counters. Add the function below to the tutorial-client.behavior namespace.

(defn total-count [_ nums] (apply + nums))

Except for the fact that it has two arguments, the first of which we are ignoring, this looks like a normal function to calculate the sum of a sequence of numbers.

The return value of a derive function will be stored at some location in the data model. The first argument to a derive function is the old value at this location. The function above always produces a new value so the old value is ignored.

The remaining arguments to the derive function depend on how that function is configured.

Derive functions are configured in the dataflow definition by adding a set of configuration vectors under the key :derive.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]}

Each derive configuration vector can have three or four elements.

[inputs output-path derive-fn input-spec] ;; input-spec is optional

In this example, a set is used to describe the inputs to this function. The function will receive the value at :my-counter and each of the :other-counter values. The output path is [:total-count] the function is total-count and the input specifier is :vals.

The input specifier describes how the arguments will be passed to the function. In this case, all of the values will be put into a single collection and passed to the function. Other options for the input specifier are:

:single-val
:map
:map-seq
:default

If you don't use an input specifier, it is the same as using :default. In this case the function will passed the inputs map from which any information can be collected.

When this derive function runs it will produce a new value in the data model under the key :total-count. We need to update the default emitter to report change at this location.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]} (app/default-emitter [:main])]

If we now refresh the Data UI, we should see the :total-count and see it update as the other counters change.

Deriving a maximum count

Next we will add a derive function to calculate the maximum counter value. To do this we will follow the exact same steps as above.

Create the derive function.

(defn maximum [old-value nums]
  (apply max (or old-value 0) nums))

Notice that this derive function makes use of the old-value.

As we did above, we need to add this derive function to the dataflow definition...

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum :vals]}

...and allow these changes to be reported.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]
   [:max-count]} (app/default-emitter [:main])]

The maximum value should now be displayed in the Data UI.

Deriving the average count

Derive functions can have inputs which are the results of other derive functions. To demonstrate this, let's create a derive function which will calculate the average of all the counters. This will use the :total-count value which has already been calculated.

(defn average-count [_ {:keys [total nums]}]
  (/ total (count nums)))

The average-count function receives a map as an argument which we destructure to get the total and nums values. To supply the arguments in this way, we use map instead of a set to configure the inputs.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum :vals]

          [{[:my-counter] :nums
            [:other-counters :*] :nums
            [:total-count] :total}
           [:average-count] average-count :map]}

Using a map to describe inputs allows us to give useful names to the keys in the argument map. In the configuration we are saying that we would like to pass the arguments to average-count as a map and we would like the keys in the map to be :nums and :total. Notice that two of the entries have the same key. This will cause all of the values of the other counters plus the value of :my-counter to be stored under the same key as a set.

Finally, add :average-count to the default-emitter.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]
   [:max-count]
   [:average-count]} (app/default-emitter [:main])]

Refreshing the page should now show the calculated average.

Simplify with a single list of counters

(defn merge-counters [_ {:keys [me others]}]
  (assoc others "Me" me))

;; derive
[{[:my-counter] :me [:other-counters] :others} [:counters] merge-counters :map]

and then we can simplify many of the derives

:derive #{[{[:my-counter] :me [:other-counters] :others} [:counters] merge-counters :map]
          [#{[:counters :*]} [:total-count] total-count :vals]
          [#{[:counters :*]} [:max-count] maximum :vals]
          [{[:counters :*] :nums [:total-count] :total} [:average-count] average-count :map]}

Dataflow

Why dataflow?

The derive functions above calculate three new values. This could all be done in transform functions. When we receive a new message we could calculate each of these values. We will see later that we would also like to send a message to a service when we update our counter.

Dataflow helps us to reduce coupling in our program. Notice that each of the derive functions above don't know anything about where the input data comes from or where the output goes. This makes the code simpler, more reusable and less likely to change.

With dataflow programming, when we need to add new features, we tend to add new functions instead of changing existing ones. This makes our code easier to maintain over time.

Because dataflow functions are small and loosely coupled to the things they depend on, they tend to more reusable.

Using dataflow allows us to write all of our behavior code as pure functions.

Next steps

We are almost finished with the application's behavior. In the next step we will see how we can make the application a bit more interesting by showing some data about how our dataflow is performing.

The tag for this step is v2.0.6.

Home | Debug Messages | Post Processing