Derived Values

Derived Values

Transform functions receive messages and apply them to the data model. What if we need to have a value in the data model that is based on values which are modified by more than one transform function?

Derive functions allow us to compute new values from any other values in the data model. The derive function will be called when any of its inputs change. Derive functions can be arranged into an arbitrary dataflow.

In this step we will calculate some additional information based on the counters that we receive. While doing this we will be introduced to the following topics:

• Derive functions
• Dataflow
• Working with dataflow inputs
• Post Processing
• debug messages

Remember to have the dev tools running before you start.

Deriving a total

We will start by calculating a total count. We would like for this total to be updated when any of the counters change.

In the namespace tutorial-client.behavior add the following derive function.

(defn total-count [_ inputs]
  (apply + (dataflow/input-vals inputs)))

A derive function has two arguments: the old value in the data model which is being updated, and the inputs. In this function we ignore the old value because the new value is always calculated based on the inputs alone.

inputs is a map which provides information about the inputs for this functions and any changes which have been made to the data model. It is hard to know what information a particular derive function will need. So instead of doing a bunch of computation which may end up being wasted work, we provide the base information and then allow you to ask for what you are interested in.

The io.pedestal.app.dataflow namespace provides a bunch of functions which allow us to extract what we want from the inputs argument.

input-map
input-vals
single-val
updated-map
added-map
remove-map
added-inputs
updated-inputs
removed-inputs

You can also write your own functions.

In the derive function above we use the input-vals function to grab the values of all of the inputs to the function.

We configure derive functions in the dataflow definition by adding a set of configuration vectors under the key :derive.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count]}

Each configuration vector has a set up inputs, the output path and the derive function to call when the inputs change. Notice that we have used a wildcard to make sure that any of the :other-coutners trigger this derive function.

When the derive function runs it will produce a new value in the data model under the key :total-count. We need to update the default emitter to report change at this location.

{:in #{[:my-counter]
       [:other-counters :*]
       [:total-count]}
 :fn (app/default-emitter :tutorial)
 :init init-emitter}

If we now refresh the Data UI, we should see the :total-count and see it update as the other counters change.

Deriving a maximum count

Next we will add a derive function to calculate the maximum counter value. To do this we will follow the exact same steps as above.

Create the derive function.

(defn maximum [old-value inputs]
  (apply max (or old-value 0) (dataflow/input-vals inputs)))

Notice that this derive function makes use of the old-value.

Add this derive function to the dataflow definition.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum]}

Allow these changes to be reported.

{:in #{[:my-counter]
       [:other-counters :*]
       [:total-count]
       [:max-count]}
 :fn (app/default-emitter :tutorial)
 :init init-emitter}

We should now see the maximum value in the Data UI.

Deriving the average count

Derive functions can have inputs which are the results of other derive functions. To demonstrate this, let's create a derive function which will calculate the average of all the counters. This will use the :total-count value which has already been calculated.

(defn average-count [_ inputs]
  (let [input-map (dataflow/input-map inputs)
        total (get input-map [:total-count])
        nums (vals (dissoc input-map [:total-count]))]
    (/ total (count nums))))

The average-count function gets the inputs as a map, extracts the total and the calculates the average. We configure this derive function as shown below.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum]
          [#{[:my-counter] [:other-counters :*] [:total-count]} [:average-count] average-count]}

And finally, add :average-count to the default-emitter.

{:in #{[:my-counter]
       [:other-counters :*]
       [:total-count]
       [:max-count]
       [:average-count]}
 :fn (app/default-emitter :tutorial)
 :init init-emitter}

Refreshing the page should now show the calculated average.

Dataflow

Why dataflow?

The derive functions above calculate three new values. This could all be done in transform functions. When we receive a new message we could calculate each of these values. We will see later that we would also like to send a message to a service when we update our counter.

Dataflow helps us to reduce coupling in our program. Notice that each of the derive functions above don't know anything about where the input data comes from or where the output goes. This makes the code simpler, more reusable and less likely to change.

With dataflow programming, when we need to add new features, we tend to add new functions instead of changing existing ones. This makes our code easier to maintain over time.

Because dataflow functions are small and loosely coupled to the things they depend on, they tend to more reusable.

Using dataflow allows us to write all of our behavior code as pure functions.

Next steps

We are almost finished the application's behavior. In the next step we will see how can make the application a bit more interesting by showing some data about how our dataflow is performing.

The tag for this step is step5.

Home | Debug Messages