Skip to content

Commit 9c29adf

Browse files
authored
Merge pull request #187 from modelix/modelql2-doc
Documentation of ModelQL v2
2 parents f9b40d6 + 405a00b commit 9c29adf

File tree

8 files changed

+256
-23
lines changed

8 files changed

+256
-23
lines changed
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
= ModelQL
2+
3+
When working with large models you will quickly run into performance issues
4+
when you try to replicate the whole model into the client.
5+
6+
While the data structure for model replication in Modelix supports partial loading of models,
7+
you still need a way to describe which data you need on the client.
8+
Loading data on demand while traversing the model also results in a poor performance,
9+
because of the potentially large number of fine-grained request.
10+
11+
A first attempt to solve this problem was to disallow lazy loading
12+
and require the client to load all required data at the beginning,
13+
before working with the model.
14+
A special query language was used to filter the data and an attempt to access a node that is not included by that query
15+
resulted in an exception, forcing the developer to adjust the query.
16+
While this results in a more predictable performance, it is also hard to maintain and still not optimal for the performance.
17+
You have to download all the data at the beginning that you might eventually need, potentially exceeding the available memory of the system.
18+
19+
The ModelQL query language provides a more dynamic way of loading parts of the model on demand,
20+
but still allows reducing the number of request to a minimum.
21+
The downside is that it's not just a different implementation hidden behind the model-api,
22+
but requires to use a different API.
23+
24+
== Reactive Streams
25+
26+
The query language is inspired by https://www.reactive-streams.org/[Reactive Streams]
27+
and the execution engine uses https://kotlinlang.org/docs/flow.html[Kotlin Flows],
28+
which is a https://kotlinlang.org/docs/coroutines-guide.html[Coroutines] compatible implementation of Reactive Streams.
29+
30+
Often it's useful to know if a stream is expected to return only one element or multiple elements.
31+
https://projectreactor.io/[Project Reactor], another implementation of Reactive Streams,
32+
introduced the notion of `Mono` and `Flux` to distinguish them.
33+
You will also find them in ModelQL.
Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
= ModelQL
2+
3+
== Independent ModelQLClient
4+
5+
ModelQL defines its own HTTP endpoint and provides server/client implementations for it.
6+
The `model-server` and the `mps-model-server-plugin` already implement this endpoint.
7+
The client can be created like this:
8+
9+
[source,kotlin]
10+
--
11+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
12+
val result: List<String?> = client.query { root ->
13+
root.children("modules").property("name").toList()
14+
}
15+
--
16+
17+
== Integration with LightModelClient
18+
19+
When creating a `LightModelClient` you can optionally provide a `ModelQLClient` instance,
20+
which allows invoking `.query { ... }` (see below) on a node returned by the `LightModelClient`.
21+
22+
[source,kotlin]
23+
--
24+
val modelqlClient = ModelQLClient.builder().build()
25+
val client = LightModelClient.builder().modelQLClient(modelqlClient).build()
26+
val result: List<String?> = client.getRootNode()!!.query {
27+
it.children("modules").property("name").toList()
28+
}
29+
--
30+
31+
== Type-safe ModelQL API
32+
33+
You can use the `model-api-gen-gradle` plugin to generate type safe extensions from your meta-model.
34+
Specify the link:../reference/component-model-api-gen-gradle.adoc#model-api-gen-gradle_attributes_modelqlKotlinDir[modelqlKotlinDir] property to enable the generation.
35+
36+
[source,kotlin]
37+
--
38+
val result: List<StaticMethodDeclaration> = client.query { root ->
39+
root.children("classes").ofConcept(C_ClassConcept)
40+
.member
41+
.ofConcept(C_StaticMethodDeclaration)
42+
.filter { it.visibility.instanceOf(C_PublicVisibility) }
43+
.toList()
44+
}
45+
--
46+
47+
== Run query on an INode
48+
49+
If a query returns a node, you can execute a new query starting from that node.
50+
51+
[source,kotlin]
52+
--
53+
val cls: ClassConcept = client.query {
54+
it.children("classes").ofConcept(C_ClassConcept).first()
55+
}
56+
val names = cls.query { it.member.ofConcept(C_StaticMethodDeclaration).name.toList() }
57+
--
58+
59+
For convenience, it's possible to access further data of that node using the https://api.modelix.org/3.6.0/model-api/org.modelix.model.api/-i-node/index.html?query=interface%20INode[INode] API,
60+
but this is not recommended though, because each access sends a new query to the server.
61+
62+
[source,kotlin]
63+
--
64+
val cls: ClassConcept = client.query {
65+
it.children("classes").ofConcept(C_ClassConcept).first()
66+
}
67+
val className = cls.name
68+
--
69+
70+
== Complex query results
71+
72+
While returning a list of elements is simple,
73+
the purpose of the query language is to reduce the number of request to a minimum.
74+
This requires combining multiple values into more complex data structures.
75+
The `zip` operation provides a simple way of doing that:
76+
77+
[source,kotlin]
78+
--
79+
val result: List<IZip3Output<Any, Int, String, List<String>>> = query { db ->
80+
db.products.map {
81+
val id = it.id
82+
val title = it.title
83+
val images = it.images.toList()
84+
id.zip(title, images)
85+
}.toList()
86+
}
87+
result.forEach { println("ID: ${it.first}, Title: ${it.second}, Images: ${it.third}") }
88+
--
89+
90+
This is suitable for combining a small number of values,
91+
but because of the missing variable names it can be hard to read for a larger number of values
92+
or even multiple zip operations assembled into a hierarchical data structure.
93+
94+
This can be solved by defining custom data classes and using the `mapLocal` operation:
95+
96+
[source,kotlin]
97+
--
98+
data class MyProduct(val id: Int, val title: String, val images: List<MyImage>)
99+
data class MyImage(val url: String)
100+
101+
val result: List<MyProduct> = remoteProductDatabaseQuery { db ->
102+
db.products.map {
103+
val id = it.id
104+
val title = it.title
105+
val images = it.images.mapLocal { MyImage(it) }.toList()
106+
id.zip(title, images).mapLocal {
107+
MyProduct(it.first, it.second, it.third)
108+
}
109+
}.toList()
110+
}
111+
result.forEach { println("ID: ${it.id}, Title: ${it.title}, Images: ${it.images}") }
112+
--
113+
114+
The `mapLocal` operation is not just useful in combination with the `zip` operation,
115+
but in general to create instances of classes only known to the client.
116+
117+
The body of `mapLocal` is executed on the client after receiving the result from the server.
118+
That's why you only have access to the output of the `zip` operation
119+
and still have to use `first`, `second` and `third` inside the query.
120+
121+
To make this even more readable there is a `buildLocalMapping` operation,
122+
which provides a different syntax for the `zip`-`mapLocal` chain.
123+
124+
[source,kotlin]
125+
--
126+
data class MyProduct(val id: Int, val title: String, val images: List<MyImage>)
127+
data class MyImage(val url: String)
128+
129+
val result: List<MyProduct> = query { db ->
130+
db.products.buildLocalMapping {
131+
val id = it.id.request()
132+
val title = it.title.request()
133+
val images = it.images.mapLocal { MyImage(it) }.toList().request()
134+
onSuccess {
135+
MyProduct(id.get(), title.get(), images.get())
136+
}
137+
}.toList()
138+
}
139+
result.forEach { println("ID: ${it.id}, Title: ${it.title}, Images: ${it.images}") }
140+
--
141+
142+
At the beginning of the `buildLocalMapping` body, you invoke `request()` on all the values you need to assemble your object.
143+
This basically adds the operand to the internal `zip` operation and returns an object that gives you access to the value
144+
after receiving it from the server.
145+
Inside the `onSuccess` block you assemble the local object using the previously requested values.
146+
147+
== Kotlin HTML integration
148+
149+
One use case of the query language is to build database applications
150+
that generate HTML pages from the data stored in the model server.
151+
You can use the https://kotlinlang.org/docs/typesafe-html-dsl.html[Kotlin HTML DSL] together with ModelQL to do that.
152+
153+
Use `buildHtmlQuery` to request data from the server and render it into an HTML string:
154+
155+
[source,kotlin]
156+
--
157+
val html = query {
158+
it.map(buildHtmlQuery {
159+
val modules = input.children("modules").requestFragment<_, FlowContent> {
160+
val moduleName = input.property("name").request()
161+
val models = input.children("models").requestFragment<_, FlowContent> {
162+
val modelName = input.property("name").request()
163+
onSuccess {
164+
div {
165+
h2 {
166+
+"Model: ${modelName.get()}"
167+
}
168+
}
169+
}
170+
}
171+
onSuccess {
172+
div {
173+
h1 {
174+
+"Module: ${moduleName.get()}"
175+
}
176+
insertFragment(models)
177+
}
178+
}
179+
}
180+
onSuccess {
181+
body {
182+
insertFragment(modules)
183+
}
184+
}
185+
})
186+
}
187+
--
188+
189+
`buildHtmlQuery` and the `requestFragment` operation are similar to the `buildLocalMapping` operation,
190+
but inside the `onSuccess` block you use the Kotlin HTML DSL.

docs/global/modules/core/pages/reference/component-model-api-gen-gradle.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ Inside of the `metamodel` block the following settings can be configured.
8282
|File
8383
|Target Kotlin directory of the generator
8484

85-
|`modelqlKotlinDir`
85+
|`modelqlKotlinDir` [[model-api-gen-gradle_attributes_modelqlKotlinDir,modelqlKotlinDir]]
8686
|File
8787
|The generation of the ModelQL API is optional, because the output has a dependency on the ModelQL runtime.
8888
If this option is set, you have to add a dependency on `org.modelix:modelql-typed`.

model-api-gen-gradle-test/kotlin-generation/src/test/kotlin/org/modelix/modelql/typed/TypedModelQLTest.kt

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ class TypedModelQLTest {
106106

107107
@Test
108108
fun simpleTest() = runTest { httpClient ->
109-
val client = ModelQLClient("http://localhost/query", httpClient)
109+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
110110
val result: Int = client.query { root ->
111111
root.children("classes").ofConcept(C_ClassConcept)
112112
.member
@@ -118,7 +118,7 @@ class TypedModelQLTest {
118118

119119
@Test
120120
fun test() = runTest { httpClient ->
121-
val client = ModelQLClient("http://localhost/query", httpClient)
121+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
122122
val result: List<Pair<String, String>> = client.query { root ->
123123
root.children("classes").ofConcept(C_ClassConcept)
124124
.member
@@ -132,7 +132,7 @@ class TypedModelQLTest {
132132

133133
@Test
134134
fun testReferences() = runTest { httpClient ->
135-
val client = ModelQLClient("http://localhost/query", httpClient)
135+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
136136
val usedVariables: Set<String> = client.query { root ->
137137
root.children("classes").ofConcept(C_ClassConcept)
138138
.member
@@ -148,7 +148,7 @@ class TypedModelQLTest {
148148

149149
@Test
150150
fun testReferencesFqName() = runTest { httpClient ->
151-
val client = ModelQLClient("http://localhost/query", httpClient)
151+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
152152
val usedVariables: Set<String> = client.query { root ->
153153
root.children("classes").ofConcept(C_ClassConcept)
154154
.member

modelql-client/src/jvmTest/kotlin/org/modelix/modelql/client/HtmlBuilderTest.kt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ class HtmlBuilderTest {
7575

7676
@Test
7777
fun modular() = runTest { httpClient ->
78-
val client = ModelQLClient("http://localhost/query", httpClient)
78+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
7979

8080
val modelTemplate = buildModelQLFragment<INode, FlowContent> {
8181
val name = input.property("name").getLater()
@@ -121,7 +121,7 @@ class HtmlBuilderTest {
121121

122122
@Test
123123
fun recursive() = runTest { httpClient ->
124-
val client = ModelQLClient("http://localhost/query", httpClient)
124+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
125125

126126
val modelTemplate = buildModelQLFragment<INode, FlowContent> {
127127
val name = input.property("name").getLater()

modelql-client/src/jvmTest/kotlin/org/modelix/modelql/client/ModelQLClientTest.kt

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ class ModelQLClientTest {
7979

8080
@Test
8181
fun test_count() = runTest { httpClient ->
82-
val client = ModelQLClient("http://localhost/query", httpClient)
82+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
8383
val result: Int = client.query { root ->
8484
root.allChildren().count()
8585
}
@@ -88,7 +88,7 @@ class ModelQLClientTest {
8888

8989
@Test
9090
fun test_properties() = runTest { httpClient ->
91-
val client = ModelQLClient("http://localhost/query", httpClient)
91+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
9292
val result: List<String?> = client.query { root ->
9393
root.children("modules").property("name").toList()
9494
}
@@ -97,7 +97,7 @@ class ModelQLClientTest {
9797

9898
@Test
9999
fun test_zip() = runTest { httpClient ->
100-
val client = ModelQLClient("http://localhost/query", httpClient)
100+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
101101
val result = client.query { root ->
102102
root.children("modules").map {
103103
it.property("name").zip(it.allChildren().nodeReference().toList())
@@ -107,7 +107,7 @@ class ModelQLClientTest {
107107

108108
@Test
109109
fun test_zipN() = runTest { httpClient ->
110-
val client = ModelQLClient("http://localhost/query", httpClient)
110+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
111111
val result = client.query { root ->
112112
root.children("modules").map {
113113
it.property("name").zip(
@@ -123,7 +123,7 @@ class ModelQLClientTest {
123123

124124
@Test
125125
fun writeProperty() = runTest { httpClient ->
126-
val client = ModelQLClient("http://localhost/query", httpClient)
126+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
127127
val updatesNodes = client.query { root ->
128128
root.children("modules")
129129
.children("models").filter { it.property("name").contains("model1a") }
@@ -148,7 +148,7 @@ class ModelQLClientTest {
148148

149149
@Test
150150
fun writeReference() = runTest { httpClient ->
151-
val client = ModelQLClient("http://localhost/query", httpClient)
151+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
152152
val updatedNodes = client.query { root ->
153153
root.children("modules")
154154
.children("models").filter { it.property("name").contains("model1a") }
@@ -167,7 +167,7 @@ class ModelQLClientTest {
167167

168168
@Test
169169
fun addNewChild() = runTest { httpClient ->
170-
val client = ModelQLClient("http://localhost/query", httpClient)
170+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
171171
val createdNodes = client.query { root ->
172172
root.children("modules")
173173
.children("models")
@@ -188,7 +188,7 @@ class ModelQLClientTest {
188188

189189
@Test
190190
fun removeNode() = runTest { httpClient ->
191-
val client = ModelQLClient("http://localhost/query", httpClient)
191+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
192192

193193
suspend fun countModels(): Int {
194194
return client.query { root ->
@@ -214,7 +214,7 @@ class ModelQLClientTest {
214214

215215
@Test
216216
fun recursiveQuery() = runTest { httpClient ->
217-
val client = ModelQLClient("http://localhost/query", httpClient)
217+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
218218

219219
val descendantsNames: IFluxUnboundQuery<INode, String?> = buildFluxQuery<INode, String?> {
220220
it.property("name") + it.allChildren().mapRecursive()
@@ -229,7 +229,7 @@ class ModelQLClientTest {
229229

230230
@Test
231231
fun testCaching() = runTest { httpClient ->
232-
val client = ModelQLClient("http://localhost/query", httpClient)
232+
val client = ModelQLClient.builder().url("http://localhost/query").httpClient(httpClient).build()
233233

234234
val result: List<Int> = client.query { root ->
235235
val numberOfNodes = root.descendants()

0 commit comments

Comments
 (0)