Skip to content

Commit 418a01e

Browse files
igorlukaninigorlukanin
authored
docs: Explain FILTER_GROUP (#7862)
1 parent 9220048 commit 418a01e

File tree

1 file changed

+343
-9
lines changed

1 file changed

+343
-9
lines changed

‎docs/pages/reference/data-model/context-variables.mdx‎

Lines changed: 343 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ You can use the following context variables within [cube][ref-ref-cubes]
44
definitions:
55

66
- [`CUBE`](#cube) for [referencing members][ref-syntax-references] of the same cube.
7-
- [`FILTER_PARAMS`](#filter_params) for optimizing generated SQL queries.
7+
- [`FILTER_PARAMS`](#filter_params) and [`FILTER_GROUP`](#filter_group) for optimizing generated SQL queries.
88
- [`SQL_UTILS`](#sql_utils) for time zone conversion.
99
- [`COMPILE_CONTEXT`](#compile_context) for creation of [dynamic data models][ref-dynamic-data-models].
1010

@@ -108,8 +108,8 @@ cubes:
108108

109109
## `FILTER_PARAMS`
110110

111-
`FILTER_PARAMS` allows you to use [filter][ref-query-filter] values during
112-
SQL generation.
111+
`FILTER_PARAMS` context variable allows you to use [filter][ref-query-filter]
112+
values from the Cube query during SQL generation.
113113

114114
This is useful for hinting your database optimizer to use a specific index
115115
or filter out partitions or shards in your cloud data warehouse so you won't
@@ -128,16 +128,49 @@ data source.
128128

129129
</WarningBox>
130130

131-
`FILTER_PARAMS` has the following syntax:
131+
`FILTER_PARAMS` has to be a top-level expression in `WHERE` and it has the
132+
following syntax:
133+
134+
<CodeTabs>
135+
136+
```yaml
137+
cubes:
138+
- name: cube_name
139+
sql: >
140+
SELECT *
141+
FROM table
142+
WHERE {FILTER_PARAMS.cube_name.member_name.filter(sql_expression)}
143+
144+
dimensions:
145+
- name: member_name
146+
# ...
147+
148+
149+
150+
```
132151

133152
```javascript
134-
FILTER_PARAMS.cube_name.member_name.filter(sql_expression)
153+
cube(`cube_name`, {
154+
sql: `
155+
SELECT *
156+
FROM table
157+
WHERE ${FILTER_PARAMS.cube_name.member_name.filter(sql_expression)}
158+
`,
159+
160+
dimensions: {
161+
member_name: {
162+
// ...
163+
}
164+
}
165+
})
135166
```
136167

137-
The `filter()` function accepts a SQL expression, which could be either
138-
a plain string or a function returning one.
168+
</CodeTabs>
169+
170+
The `filter()` function accepts `sql_expression`, which could be either
171+
a string or a function returning a string.
139172

140-
### String
173+
### Example with string
141174

142175
See the example below for the case when a string is passed to `filter()`:
143176

@@ -216,7 +249,7 @@ WHERE
216249
}
217250
```
218251

219-
### Function
252+
### Example with function
220253

221254
You can also pass a function as a `filter()` argument. This way, you can
222255
add BigQuery shard filtering, which will reduce your billing cost.
@@ -277,6 +310,306 @@ type conversions in this case.
277310

278311
</InfoBox>
279312

313+
## `FILTER_GROUP`
314+
315+
If you use `FILTER_PARAMS` in your query more than once, you must wrap them
316+
with `FILTER_GROUP`.
317+
318+
<WarningBox>
319+
320+
Otherwise, if you combine `FILTER_PARAMS` with any logical operators other than
321+
`AND` in SQL or if you use filters with [boolean operators][ref-filter-boolean]
322+
in your Cube queries, incorrect SQL might be generated.
323+
324+
</WarningBox>
325+
326+
`FILTER_GROUP` has to be a top-level expression in `WHERE` and it has the
327+
following syntax:
328+
329+
<CodeTabs>
330+
331+
```yaml
332+
cubes:
333+
- name: cube_name
334+
sql: >
335+
SELECT *
336+
FROM table
337+
WHERE {FILTER_GROUP(
338+
FILTER_PARAMS.cube_name.member_name.filter(sql_expression),
339+
FILTER_PARAMS.cube_name.another_member_name.filter(sql_expression)
340+
)}
341+
342+
dimensions:
343+
- name: member_name
344+
# ...
345+
346+
- name: another_member_name
347+
# ...
348+
349+
350+
351+
352+
```
353+
354+
```javascript
355+
cube(`cube_name`, {
356+
sql: `
357+
SELECT *
358+
FROM table
359+
WHERE ${FILTER_GROUP(
360+
FILTER_PARAMS.cube_name.member_name.filter(sql_expression),
361+
FILTER_PARAMS.cube_name.another_member_name.filter(sql_expression)
362+
)}
363+
`,
364+
365+
dimensions: {
366+
member_name: {
367+
// ...
368+
},
369+
370+
another_member_name: {
371+
// ...
372+
}
373+
}
374+
})
375+
```
376+
377+
</CodeTabs>
378+
379+
### Example
380+
381+
To understand the value of `FILTER_GROUP`, consider the following data model
382+
where two `FILTER_PARAMS` are combined in SQL using the `OR` operator:
383+
384+
<CodeTabs>
385+
386+
```yaml
387+
cubes:
388+
- name: filter_group
389+
sql: >
390+
SELECT *
391+
FROM (
392+
SELECT 1 AS a, 3 AS b UNION ALL
393+
SELECT 2 AS a, 2 AS b UNION ALL
394+
SELECT 3 AS a, 1 AS b
395+
) AS data
396+
WHERE
397+
{FILTER_PARAMS.filter_group.a.filter("a")} OR
398+
{FILTER_PARAMS.filter_group.b.filter("b")}
399+
400+
dimensions:
401+
- name: a
402+
sql: a
403+
type: number
404+
405+
- name: b
406+
sql: b
407+
type: number
408+
409+
410+
411+
412+
```
413+
414+
```javascript
415+
cube(`filter_group`, {
416+
sql: `
417+
SELECT *
418+
FROM (
419+
SELECT 1 AS a, 3 AS b UNION ALL
420+
SELECT 2 AS a, 2 AS b UNION ALL
421+
SELECT 3 AS a, 1 AS b
422+
) AS data
423+
WHERE
424+
${FILTER_PARAMS.filter_group.a.filter('a')} OR
425+
${FILTER_PARAMS.filter_group.b.filter('b')}
426+
`,
427+
428+
dimensions: {
429+
a: {
430+
sql: `a`,
431+
type: `number`
432+
},
433+
434+
b: {
435+
sql: `b`,
436+
type: `number`
437+
}
438+
}
439+
})
440+
```
441+
442+
</CodeTabs>
443+
444+
If the following query is run...
445+
446+
```json
447+
{
448+
"dimensions": [
449+
"filter_group.a",
450+
"filter_group.b"
451+
],
452+
"filters": [
453+
{
454+
"member": "filter_group.a",
455+
"operator": "gt",
456+
"values": ["1"]
457+
},
458+
{
459+
"member": "filter_group.b",
460+
"operator": "gt",
461+
"values": ["1"]
462+
}
463+
]
464+
}
465+
```
466+
467+
...the following (logically incorrect) SQL will be generated:
468+
469+
```sql
470+
SELECT
471+
"filter_group".a,
472+
"filter_group".b
473+
FROM (
474+
SELECT *
475+
FROM (
476+
SELECT 1 AS a, 3 AS b UNION ALL
477+
SELECT 2 AS a, 2 AS b UNION ALL
478+
SELECT 3 AS a, 1 AS b
479+
) AS data
480+
WHERE
481+
(a > 1) OR -- Incorrect logical operator here
482+
(b > 1)
483+
) AS "filter_group"
484+
WHERE
485+
"filter_group".a > 1 AND
486+
"filter_group".b > 1
487+
GROUP BY 1, 2
488+
```
489+
490+
As you can see, since an array of filters has `AND` semantics, Cube has
491+
correctly used the `AND` operator in the "outer" `WHERE`. At the same time,
492+
the hardcoded `OR` operator has propagated to the "inner" `WHERE`, leading to
493+
a logically incorrect query.
494+
495+
Now, if the cube is defined the following way...
496+
497+
<CodeTabs>
498+
499+
```yaml
500+
cubes:
501+
- name: filter_group
502+
sql: >
503+
SELECT *
504+
FROM (
505+
SELECT 1 AS a, 3 AS b UNION ALL
506+
SELECT 2 AS a, 2 AS b UNION ALL
507+
SELECT 3 AS a, 1 AS b
508+
) AS data
509+
WHERE
510+
{FILTER_GROUP(
511+
FILTER_PARAMS.filter_group.a.filter("a"),
512+
FILTER_PARAMS.filter_group.b.filter("b")
513+
)}
514+
515+
# ...
516+
```
517+
518+
```javascript
519+
cube(`filter_group`, {
520+
sql: `
521+
SELECT *
522+
FROM (
523+
SELECT 1 AS a, 3 AS b UNION ALL
524+
SELECT 2 AS a, 2 AS b UNION ALL
525+
SELECT 3 AS a, 1 AS b
526+
) AS data
527+
WHERE
528+
${FILTER_GROUP(
529+
FILTER_PARAMS.filter_group.a.filter('a'),
530+
FILTER_PARAMS.filter_group.b.filter('b')
531+
)}
532+
`,
533+
534+
// ...
535+
```
536+
537+
</CodeTabs>
538+
539+
...the following correct SQL will be generated for the same query:
540+
541+
```sql
542+
SELECT
543+
"filter_group".a,
544+
"filter_group".b
545+
FROM (
546+
SELECT *
547+
FROM (
548+
SELECT 1 AS a, 3 AS b UNION ALL
549+
SELECT 2 AS a, 2 AS b UNION ALL
550+
SELECT 3 AS a, 1 AS b
551+
) AS data
552+
WHERE
553+
(a > 1) AND -- Correct logical operator here
554+
(b > 1)
555+
) AS "filter_group"
556+
WHERE
557+
"filter_group".a > 1 AND
558+
"filter_group".b > 1
559+
GROUP BY 1, 2
560+
```
561+
562+
You can also use [boolean operators][ref-filter-boolean] in the Cube query
563+
to express more complex filtering logic:
564+
565+
```json
566+
{
567+
"dimensions": [
568+
"filter_group.a",
569+
"filter_group.b"
570+
],
571+
"filters": [
572+
{
573+
"or": [
574+
{
575+
"member": "filter_group.a",
576+
"operator": "gt",
577+
"values": ["1"]
578+
},
579+
{
580+
"member": "filter_group.b",
581+
"operator": "gt",
582+
"values": ["1"]
583+
}
584+
]
585+
}
586+
]
587+
}
588+
```
589+
590+
With `FILTER_GROUP`, the following correct SQL will be generated:
591+
592+
```sql
593+
SELECT
594+
"filter_group".a,
595+
"filter_group".b
596+
FROM (
597+
SELECT *
598+
FROM (
599+
SELECT 1 AS a, 3 AS b UNION ALL
600+
SELECT 2 AS a, 2 AS b UNION ALL
601+
SELECT 3 AS a, 1 AS b
602+
) AS data
603+
WHERE
604+
(a > 1) OR
605+
(b > 1)
606+
) AS "filter_group"
607+
WHERE
608+
"filter_group".a > 1 OR
609+
"filter_group".b > 1
610+
GROUP BY 1, 2
611+
```
612+
280613
## `SQL_UTILS`
281614
282615
### `convertTz`
@@ -464,3 +797,4 @@ cube(`orders`, {
464797
[ref-dynamic-data-models]: /product/data-modeling/dynamic/jinja
465798
[ref-query-filter]: /product/apis-integrations/rest-api/query-format#query-properties
466799
[ref-dynamic-jinja]: /product/data-modeling/dynamic/jinja
800+
[ref-filter-boolean]: /product/apis-integrations/rest-api/query-format#boolean-logical-operators

0 commit comments

Comments
 (0)