Learning Aardvark.Media #1
This page is a central hub linking to all information necessary to program within the Aardvark platform, e.g. language resources, tutorials, and installation guides.
- What is aardvark media?
- What does it do?
- What it doesn't do?
Aardvark is mainly programmed in F#, a functional programming language. Hence, it is often said one has to 'unlearn' imperative programming styles and object oriented thinking. F# for Fun and Profit gives a great and complete overview of F# and also elaborates on why to use it. This is not about learning a new language, this is about evolving a new way to think, a new way to approach problems, and only ultimately expressing this in a new programming language.
Good starting points are the following two talks functional design principles and a functional approach to domain driven design. After getting an idea about the functional paradigms you can dive more into the syntax and language functions covered in the written series.
For programming web-based GUI elements within the Aardvark platform, it is important to know your way around web technologies, such as, HTML, CSS, SVG, and JavaScript. There is no really useful complete tutorial resource for our needs, so we will explain these parts by example in later sections. For most of our visible UI parts we found Semantic UI very useful, but you could use any other UI library together with Aardvark.Media.
The execution model of every App in Aardvark.Media follows the same pattern of unidirectional data flow, known as the ELM architecture (see ELM). It is most natural to start with the Model, which shall hold our data. The View describes how this data is visualized. In Aardvark.Media this is either a DomNode (part of an HTML page) or a SceneGraph node for 3D real-time rendering. The user can interact with what the view produces, e.g. clicking on a button or picking a 3D object, which in turn can generate actions. These actions strive to change our data, e.g. the color of an object the user selects in a colorpicker. The Update logic specifies how these actions change our data and generate a new model (immutable data). What we call an App is simply a combination of Model-Update-View and its Actions.
Apps exist at different granularities. There is an app for a numeric data field while a whole project is also just and app. Larger, more complex apps are simply composed together from smaller apps. It is quite natural that complex model types consist of more primitive model types and ultimately primitive datatypes on the lowest level. In later sections we will see how views, actions and update functions can easily be composed together to build powerful and maintainable apps.
The following example shall lead us through all topics relevant to building powerful apps in Aardvark.Media combining 3D graphics and a web-based GUI in a declarative and functional way. We will render a couple of 3D objects and transform them with controls we write from scratch. We demonstrate how the Model-Update-View paradigma works for generating GUI and 3D views and how to build powerful apps by composition. In the later chapters we we will demonstrate how our mod system and adaptive data structures handle change and are essential to tame complexity in larger apps.
First, we will build a numeric control, which lets the users increment and decrement an float value. We will approach this problem in the sequence of Model-Update-View, starting with the model.
We simply build a record type holding our float value. For future use, we also attribute it with 'DomainType'.
[<DomainType>]
type Model = { value : float }Specifying update is a two-fold effort of first defining actions of what does our app do to the model ...
type Action =
| Increment
| Decrement... and second of defining how do these actions affect our model.
let update (m : Model) (a : Action) =
match a with
| Increment -> { m with value = m.value + 0.1 }
| Decrement -> { m with value = m.value - 0.1 } The typical signature of update is Model->Action->Model, taking the current model and an action and returning an updated version of our model. Although, actions can in principle come from anywhere our actions for now will originate from our app's UI, which we specify in the view function. Consequently, the view function defines how our model is visualized and how the user can manipulate the model via UI elements.
First, we create two buttons , one for incrementing and one for decrementing our model's value.
button [clazz "ui button"; onClick (fun _ -> Increment)] [text "+"]
button [clazz "ui button"; onClick (fun _ -> Decrement)] [text "-"]Besides the architecture of ELM we were also inspired by ELMs way of constructing DOM (Document Object Model) elements, such as divs, buttons, and unordered lists. This allows us to emit HTML code for our views directly from F# code. The code above roughly equivalents to:
<button class="ui button" onClick=Increment()>+</button>
<button class="ui button" onClick=Decrement()>-</button>You may notice that we write the 'class' attribute as 'clazz'. This is not because we are the cool kids, but naturally 'class' is a reserved keyword in F#. Most of the html and Aardvark.UI tags are 'elems' consisting of an attribute list and a child list, while 'voidelems' such as 'br' only have attributes. A list of all available tags can be found here. The finished view function, also containing a visualization for our model, looks like the following:
let view (m : MModel) =
require Html.semui (
body [] (
[
div [] [
button [clazz "ui button"; onClick (fun _ -> Increment)] [text "+"]
button [clazz "ui button"; onClick (fun _ -> Decrement)] [text "-"]
br []
text "my value:"
br []
Incremental.text (m.value |> Mod.map(fun x -> sprintf "%.1f" x))
]
]
)
)'require Html.semui' enables the emitted html code to refer to web ressources such as css style sheets or javascript libraries. In this case we emit a script reference for the UI styling library Semantic UI. For now we ignore MModel, Incremental.text, and Mod.map which will be discussed in later sections.
The final lines of every app is putting the individual parts together...
let app =
{
unpersist = Unpersist.instance //ignore for now
threads = fun _ -> ThreadPool.empty //ignore for now
initial = { value = 0.0 }
update = update
view = view
}
let start() = App.start app...resulting in the following output:
Our vector control shall allow us to manipulate XYZ of a 3D vector. Naturally, we want to reuse our previous example and just combine three numeric controls via composition. First, we create a composed VectorModel by using a record type consisting of individual NumericModels for X,Y and Z.
[<DomainType>]
type VectorModel = {
x : NumericModel
y : NumericModel
z : NumericModel
}Our composed app now has a composed model and of course needs a composed action describing what our VectorControl can do. If we come accross an action in this control either X,Y or Z is incremented or decremented. However, increment and decrement are already captured in the NumericControl.Action, so we don't want to specify that again. Since an action is either an update on x or on y or on z our composed action is a union type.
type Action =
| UpdateX of NumericControl.Action
| UpdateY of NumericControl.Action
| UpdateZ of NumericControl.ActionLooking at Model and Action, our VectorControl is quite limited, it can't do anything by itself really. Therefore, our update funciton just simply calls update functions from the NumericControl.
let update (m : VectorModel) (a : Action) =
match a with
| UpdateX a -> { m with x = NumericControl.update m.x a }
| UpdateY a -> { m with y = NumericControl.update m.y a }
| UpdateZ a -> { m with z = NumericControl.update m.z a }The results are handed to the relevant part of our composed model, e.g. the X coordinate. The match on UpdateX unpacks the composed action a and we can directly pass a NumericControl.Action to update. Update returns a new NumericModel which we then use to update the VectorModel.
Such pure composition apps are quite frequent, for instance when making a property control consisting of numeric, bool, and string values, or when making a sidebar app consisting of multiple property apps. But more often composing apps add their own actions. In this case we could add Normalize and Reset to our VectorControl.
type Action =
| UpdateX of NumericControl.Action
| UpdateY of NumericControl.Action
| UpdateZ of NumericControl.Action
| Normalize
| Reset
let update (m : VectorModel) (a : Action) =
match a with
| UpdateX a -> { m with x = NumericControl.update m.x a }
| UpdateY a -> { m with y = NumericControl.update m.y a }
| UpdateZ a -> { m with z = NumericControl.update m.z a }
| Normalize ->
let v = V3d(m.x.value,m.y.value,m.z.value)
v.Normalized |> toVectorModel
| Reset -> VectorModel.initialBefore starting with the view, we will adapt the viewing code of the NumericControl a bit. In the way we programmed it above it is not very suitable to be stacked together into a VectorControl. That being said, a composing app is responsible for how the composed parts should be viewed together.
let view' (m : MNumericModel) =
require Html.semui (
table [][
tr[] [
td[] [a [clazz "ui label circular Big"] [Incremental.text (m.value |> Mod.map(fun x -> " " + x.ToString()))]]
td[] [
button [clazz "ui icon button"; onClick (fun _ -> Increment)] [text "+"]
button [clazz "ui icon button"; onClick (fun _ -> Decrement)] [text "-"]
]
]
]
)This is our updated view function view', which uses an html table to align things more neatly. The result looks like this:
Our VectorControl's view function should now call the individual NumericControl view functions and put them in a nice table layout.
let view (m : MVectorModel) =
require Html.semui (
div[][
table [] [
tr[][
td[][a [clazz "ui label circular Big"][text "X:"]]
td[][NumericControl.view' m.x |> UI.map UpdateX]
]
tr[][
td[][a [clazz "ui label circular Big"][text "Y:"]]
td[][NumericControl.view' m.y |> UI.map UpdateY]
]
tr[][
td[][a [clazz "ui label circular Big"][text "Z:"]]
td[][NumericControl.view' m.z |> UI.map UpdateZ]
]
tr[][
td[attribute "colspan" "2"][
div[clazz "ui buttons small"][
button [clazz "ui button"; onClick (fun _ -> Normalize)] [text "Norm"]
button [clazz "ui button"; onClick (fun _ -> Reset)] [text "Reset"]
]
]
]
]
]
)Everything looks quite familiar, except for UI.map. The view function of the VectorControl is supposed to return DomNode, while NumericControl.view' returns a DomNode<NumericControl.Action> resulting in a type mismatch. UI.map resolves this problem by lifting the action type of the subapp to the action type of the composing app by simply mapping it to the respective composing action.
The deep nesting of domnodes results from the quite cumbersome way to specify tables in html. Further, the vector control is probably not the most elegant control you have ever seen. However, we will stick to the this level of verbosity for explanatory reasons. Using the UI primitives of Aardvark.Media results in something more like this:
Html.SemUi.accordion "Scale" "Expand" true [
div [] [Vector3d.view model.scale |> UI.map V3dMessage]
]
In the described implementation we used composition to build a vector control by just reusing our numeric control code. In the same way we can compose a transformation app for manipulating 3 vectors for translation, rotation, and scale. Instead of implementing this step by step we use it to revisit the composition illustration from earlier filling it with actual code.
In this chapter we covered how the meaning of model-update-view and what apps look like in Aardvark.Media. We built a small app from scratch and composed a more powerful app from that. We further illustrated how to use domnodes which ultimately emit html and javascript code for our GUI.
Before expanding our example with a 3D view we will revisit the ELM architecture and cover incremental evaluation.
When naively implementing the ELM architecture illustrated above we execute the whole view function on every update of the model. This does not scale well with larger DOM trees, larger SceneGraphs, nor high frequency updates, such as camera movement. We actually want to only update those parts of the scene which have actually changed - so we want incremental updates.
Aardvark.Rendering is a powerful rendering engine which supports incremental updates of 3D scenes, while Aardvark.Media does the same thing for DomNodes. We will not go into details here (refer to publications) but at the heart of all this is the Mod System. Instead of specifying on how to react to change explicitly a dependency is created between a data field in the model and a visual representation, e.g. a color : C4b and the rendering color of an Sg.Cube.
Within our nice unidirectional dataflow we don't want to use the mod system directly. Therefore, a compile step generates an MModel from our Model, translating every data field to a mod field which is aware of being changed. Most Sg nodes take mod version and for GUI elements there are incremental tags, such as Incremental.Text taking an IMod.
Here the incremental loop is illustrated with actual code. Our model contains a 3D vector (V3d) specifying a scale. Our compiler, the diff generator, generates an MModel wrapping the V3d into a mod. Our 3D view shows a cube which is transformed and further triggers a reset action on double click. Sg.trafo takes Mod. To generate a Mod we use Mod.map and speficy a mapping Mod -> Mod. The Sg view and the domnode view can both trigger a reset action which changes our model. This change is reflected in an update of MModel.value which only triggers a reevaluation of Sg.trafo.
-
mods
-
adaptive
-
MModels for 3D view
-
MModels for GUI views (AttributeMaps)
-
adaptive pendants of datastructures (alist, aset, amap, ...)