Learning Aardvark.Media #2

Adaptive Datastructures

In the previous section we only dealt with very limited scene modifications, namely the change of a single value or property e.g. the scale of a box. In this section we will focus on how to handle structural changes to our scene. Our leading example will be the BoxSelectionDemo, availabe in Aardvark.Media when loading the Aardvark.Media.Examples.sln` solution. Our application can add and remove boxes and further offers interaction to select and hover boxes in 3D as well as in a GUI list view. Adaptive datastructures are crucial for dealing with structural change, but we will also cover other relevant topics along the way, such as integrating a camera controller.

Compilation Scheme

When recalling the previous chapter, we got to know two types that 'react to change', AVal<'a> for wrapping a value into an AVal and the adaptive type AdaptiveType provided by the diffgenerator through the [ModelType] attribute. Since functional programming is all about data structures we also need adaptive versions of collections such as lists, sequences, or maps.

Immutable	Adaptive	Remarks
α(IndexList<'T>)	alist<α('T)>	alist<aval<'T>> -> alist<'T>
α(HashSet<'T>)	aset<α('T)>	aset<aval<'T>> -> aset<'T>
α(HashMap<'K, 'V>)	amap<'K, α('V)>	amap<'K, aval<'V>> -> amap<'K, 'V>
α({ a:'T;.. })	{ a:α('T);..}	product types
α('T * ..)	(α('T) * ..)	tuples
α(struct('T * ..))	(α('T) * ..)	struct tuples
α(|A of 'T..)	aval<|AdaptiveA of α('T)..>	sum types
α('T)	aval<'T>	opaque types

This table shows the compilation scheme of the adaptify service, telling us which type is mapped to which adaptive type, for instance, an IndexList in our domain type Abc will be an alist in our adaptive type AdaptiveAbc. Adaptive datastructures are part of FSharp.Data.Adaptive and the adaptify service is creating them from immutable types. Both links contain extensive example and tutorial ressources supporting a deeper understanding of these mechanics.

3D rendering control

Back to our BoxSelectionApp, we start again with the model

[<ModelType>]
type BoxSelectionDemoModel = {
    boxes : IndexList<Box3d>
}

containing an alist of Box3d. In AdaptiveBoxSelectionDemoModel this will be accessible for our view function as alist<Box3d>. We want our 3D view and our GUI, a list of boxes, to react to changes of this list, e.g. add a new 3D cube if a box is added. alist not only reacts to structural changes, but also to changes to the properties of each element efficiently propagated via the dependency graph.

[<ModelType>]
type VisibleBox = {
    geometry : Box3d
    color    : C4b    

    [<NonAdaptive>]
    id       : string
}

[<ModelType>]
type BoxSelectionDemoModel = {
    camera     : CameraControllerState    
    boxes      : IndexList<VisibleBox>    
    boxHovered : option<string>    
}

To make our naked box geometries a bit more interesting we wrap them into VisibleBox which also holds a color and an id. The attribute [<NonAdaptive>] tells adaptify to not compile this field to an AVal. Of course, we want our ids constant, otherwise it will be difficult to identify our boxes.

As you can see, we also included a CameraControllerState into our model. To have an interactive 3D view, we simply compose a CameraControllerApp into our app starting with just adding its model, which should rather be called CameraControllerModel if we were a 100% strict with the naming. Next, we wrap the camera controller actions into our BoxSelection actions and handle the action in the update function, as with any other composition.

type Action =
| CameraMessage of CameraControllerMessage
...
    
let update (model : BoxSelectionDemoModel) (act : Action) =
    match act with
    | CameraMessage m -> 
        { model with camera = CameraController.update model.camera m }

Finally, we embed the CameraController into our view function. The CameraController's view is exposed as an incremental control. On second thought, this is just another view function taking a model, an action, and some additonal attributes. Most notable here are the AttributeMap and the child nodes as last argument.

 let view (model : AdaptiveBoxSelectionDemoModel) =                                   
        let frustum = Mod.constant (Frustum.perspective 60.0 0.1 100.0 1.0)
              
        div [clazz "ui"; style "background: #1B1C1E"] [
            CameraController.controlledControl model.camera CameraMessage frustum
                (AttributeMap.ofList [
                    attribute "style" "width:65%; height: 100%; float: left;"
                ])
                (
                    Sg.box (AVal.constant C4b.Gray) (AVal.constant Box3d.Unit)
                    |> Sg.shader {
                        do! DefaultSurfaces.trafo
                        do! DefaultSurfaces.vertexColor
                        do! DefaultSurfaces.simpleLighting
                    }                
                    |> Sg.requirePicking
                    |> Sg.noEvents                       
                )
        ]

AttributeMaps are a special way of defining changeable attributes for a DomNode. In the above case we us static attributes for an incremental DomNode, so we can just use static syntax and convert it to an AttributeMap via AttributeMap.ofList. The child node parameter of the controlledControl function is of the type ISg<'msg>, in our case a static grey box not capable of triggering any actions. Most interestingly the output of controlledControl is a DomNode of Action, since the 3D rendering frames are written into an image. Therefore, the result of our controlledControl function composes with any other DomNodes, just as a div or a text would. Our interactive intermediate result looks like this:

ISg of Action

Before dealing with multiple boxes we think of the actions we want to support and implement them for a single box.

type Action =
| CameraMessage    of CameraControllerMessage
| Select           of string     
| ClearSelection
| HoverIn          of string 
| HoverOut                     
| AddBox
| RemoveBox    

We want to click on a box and Select it resulting in a persistent highlighting (color red) or a peek highlighting on HoverIn (color blue) which stops on HoverOut. Similar to attaching actions to DomNodes (onclick button) we can specify which actions are triggered when clicking on a Sg.Box.

Sg.box color box.geometry
|> Sg.shader {
    do! DefaultSurfaces.trafo
    do! DefaultSurfaces.vertexColor
    do! DefaultSurfaces.simpleLighting
    }                
|> Sg.requirePicking
|> Sg.noEvents
|> Sg.withEvents [
    Sg.onClick (fun _  -> Select box.id)
    Sg.onEnter (fun _  -> Enter  box.id)
    Sg.onLeave (fun () -> Exit)
]

Since we need to know which box has been selected/hovered later on we send the box.id with the Action into the update function. For simplicity we will we will only go through the hovering interaction here.

let update (model : BoxSelectionDemoModel) (act : Action) =
    match act with
    | CameraMessage m -> 
        { model with camera = CameraController.update model.camera m }
    | HoverIn id -> { model with boxHovered = Some id }            
    | HoverOut   -> { model with boxHovered = None }        
    | _          -> model

We map our hovering interaction to the option of boxid. Either, there is an id or no hovered has taken place. Now we want to react on this changing hovering state by changing the cubes color. adaptify gives us an AVAl<Option<string>> in the view function, while we need an AVal<C4b>. Once again, we want to just map things from AVal<'a> to AVal<'b> using AVal.map.

let color = 
    model.boxHovered 
    |> Mod.map (fun x -> 
        match x with
        | Some k -> if k = "box" then C4b.Blue else C4b.Gray
        | None -> C4b.Gray)

open topics

• Incremental Domnodes and Attribute maps
• performance implications
• BoxSelection Demo

continue with Aardvark.Media #3 or back to Aardvark.Media #1