Skip to content

Commit c3ddf1d

Browse files
BlargianBlargian
committed
add note about new multiple filtering on projections
1 parent fcb8b14 commit c3ddf1d

File tree

1 file changed

+14
-14
lines changed

1 file changed

+14
-14
lines changed

‎docs/data-modeling/projections/2_materialized-views-versus-projections.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -16,20 +16,20 @@ may want to pick one over the other in certain scenarios.
1616

1717
The table below summarizes the key differences between materialized views and projections for various aspects of consideration.
1818

19-
| Aspect | Materialized views | Projections |
20-
|------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
21-
| Data storage and location | Store their results in a **separate, explicit target table** on insert to a source table, acting as insert triggers. | Projections create optimized data layouts that are physically **stored alongside the main table data** and are invisible to the user. |
22-
| Update mechanism | Operate **synchronously** on `INSERT` to the source table (for incremental materialized views). Note: they can also be **scheduled** using refreshable materialized views. | **Asynchronous** updates in the background upon `INSERT` to the main table. |
23-
| Query interaction | Working with Materialized Views requires querying the **target table directly**, meaning that users need to be aware of the existence of materialized views when writing queries. | Projections are **automatically selected** by ClickHouse's query optimizer, and are transparent in the sense that the user does not have to modify their queries to the table with the projection in order to utilise it. |
24-
| Handling `UPDATE` / `DELETE` | **Do not automatically react** to `UPDATE` or `DELETE` operations on the source table as materialized views have no knowledge of the source table, acting only as insert triggers _to_ a source table. This can lead to potential data staleness between source and target tables and requires workarounds or periodic full refresh. (via Refreshable materialized view). | By default, are **incompatible with `DELETED` rows** (especially lightweight deletes). `lightweight_mutation_projection_mode` (v24.7+) can enable compatibility. |
25-
| `JOIN` support | Yes. Refreshable materialized views can be used for complex denormalization. Incremental materialized views only trigger on left-most table inserts. | No. `JOIN` operations are not supported within projection definitions for filtering the materialised data. |
26-
| `WHERE` clause in definition | Yes. `WHERE` clauses can be included to filter data before materialization. | No. `WHERE` clauses are not supported within projection definitions for filtering the materialized data. |
27-
| Chaining capabilities | Yes, the target table of one materialized view can be the source for another materialized view, enabling multi-stage pipelines. | No. Projections cannot be chained. |
28-
| Applicable table engines | Can be used with various source table engines, but target tables are usually of the `MergeTree` family. | **Only available** for `MergeTree` family table engines. |
29-
| Failure handling | Failure during data insertion means that data is lost in the target table, leading to potential inconsistency. | Failures are handled **silently** in the background. Queries can seamlessly mix materialized and unmaterialized parts. |
30-
| Operational overhead | Requires explicit target table creation and often manual backfilling. Managing consistency with `UPDATE`/`DELETE` increases complexity. | Projections are automatically maintained and kept-in-sync and generally have a lower operational burden. |
31-
| `FINAL` query compatibility | Generally compatible, but often require `GROUP BY` on the target table. | **Do not work** with `FINAL` queries. |
32-
| Lazy materialization | Yes. | Monitor for projection compatibility issues when using materialization features. You may need to set `query_plan_optimize_lazy_materialization = false` |
19+
| Aspect | Materialized views | Projections |
20+
|------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
21+
| Data storage and location | Store their results in a **separate, explicit target table** on insert to a source table, acting as insert triggers. | Projections create optimized data layouts that are physically **stored alongside the main table data** and are invisible to the user. |
22+
| Update mechanism | Operate **synchronously** on `INSERT` to the source table (for incremental materialized views). Note: they can also be **scheduled** using refreshable materialized views. | **Asynchronous** updates in the background upon `INSERT` to the main table. |
23+
| Query interaction | Working with Materialized Views requires querying the **target table directly**, meaning that users need to be aware of the existence of materialized views when writing queries. | Projections are **automatically selected** by ClickHouse's query optimizer, and are transparent in the sense that the user does not have to modify their queries to the table with the projection in order to utilise it. From version 25.6 it is also possible to filter by more than one projection. |
24+
| Handling `UPDATE` / `DELETE` | **Do not automatically react** to `UPDATE` or `DELETE` operations on the source table as materialized views have no knowledge of the source table, acting only as insert triggers _to_ a source table. This can lead to potential data staleness between source and target tables and requires workarounds or periodic full refresh. (via Refreshable materialized view). | By default, are **incompatible with `DELETED` rows** (especially lightweight deletes). `lightweight_mutation_projection_mode` (v24.7+) can enable compatibility. |
25+
| `JOIN` support | Yes. Refreshable materialized views can be used for complex denormalization. Incremental materialized views only trigger on left-most table inserts. | No. `JOIN` operations are not supported within projection definitions for filtering the materialised data. |
26+
| `WHERE` clause in definition | Yes. `WHERE` clauses can be included to filter data before materialization. | No. `WHERE` clauses are not supported within projection definitions for filtering the materialized data. |
27+
| Chaining capabilities | Yes, the target table of one materialized view can be the source for another materialized view, enabling multi-stage pipelines. | No. Projections cannot be chained. |
28+
| Applicable table engines | Can be used with various source table engines, but target tables are usually of the `MergeTree` family. | **Only available** for `MergeTree` family table engines. |
29+
| Failure handling | Failure during data insertion means that data is lost in the target table, leading to potential inconsistency. | Failures are handled **silently** in the background. Queries can seamlessly mix materialized and unmaterialized parts. |
30+
| Operational overhead | Requires explicit target table creation and often manual backfilling. Managing consistency with `UPDATE`/`DELETE` increases complexity. | Projections are automatically maintained and kept-in-sync and generally have a lower operational burden. |
31+
| `FINAL` query compatibility | Generally compatible, but often require `GROUP BY` on the target table. | **Do not work** with `FINAL` queries. |
32+
| Lazy materialization | Yes. | Monitor for projection compatibility issues when using materialization features. You may need to set `query_plan_optimize_lazy_materialization = false` |
3333

3434
## Comparing materialized views and projections {#choose-between}
3535

0 commit comments

Comments
 (0)