Skip to content

Commit 219eef3

Browse files
committed
formatting and language edit
1 parent 5ab3d46 commit 219eef3

File tree

1 file changed

+24
-2
lines changed

1 file changed

+24
-2
lines changed

docs/en/cloud/bestpractices/avoidoptimizefinal.md

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,32 @@
22
slug: /en/cloud/bestpractices/avoid-optimize-final
33
sidebar_label: Avoid Optimize Final
44
title: Avoid Optimize Final
5+
keywords: ['OPTIMIZE TABLE', 'FINAL', 'unscheduled merge']
56
---
67

7-
Using the [`OPTIMIZE TABLE ... FINAL`](/docs/en/sql-reference/statements/optimize/) query will initiate an unscheduled merge of data parts for the specific table into one data part. During this process, ClickHouse reads all the data parts, uncompresses, merges, compresses them into a single part, and then rewrites back into object store, causing huge CPU and IO consumption.
8+
Using the [`OPTIMIZE TABLE ... FINAL`](/docs/en/sql-reference/statements/optimize/) query initiates an unscheduled merge of data parts for a specific table into one single data part.
9+
During this process, ClickHouse performs the following steps:
10+
11+
- Data parts are read.
12+
- The parts get uncompressed.
13+
- The parts get merged.
14+
- They are compressed into a single part.
15+
- The part is then written back into the object store.
16+
17+
The operations described above are resource intensive, consuming significant CPU and disk I/O.
18+
It is important to note that using this optimization will force a rewrite of a part,
19+
even if merging to a single part has already occurred.
20+
21+
Additionally, use of the `OPTIMIZE TABLE ... FINAL` query may disregard
22+
setting [`max_bytes_to_merge_at_max_space_in_pool`](https://clickhouse.com/docs/en/operations/settings/merge-tree-settings#max-bytes-to-merge-at-max-space-in-pool) which controls the maximum size of parts
23+
that ClickHouse will typically merge by itself in the background.
24+
25+
The [`max_bytes_to_merge_at_max_space_in_pool`](https://clickhouse.com/docs/en/operations/settings/merge-tree-settings#max-bytes-to-merge-at-max-space-in-pool) setting is by default set to 150 GB.
26+
When running `OPTIMIZE TABLE ... FINAL`,
27+
the steps outlined above will be performed resulting in a single part after merge.
28+
This remaining single part could exceed the 150 GB specified by the default of this setting.
29+
This is another important consideration and reason why you should avoid use of this statement,
30+
since merging a large number of 150 GB parts into a single part could require a significant amount of time and/or memory.
831

9-
Note that this optimization rewrites the one part even if they are already merged into a single part. Also, it is important to note the scope of a "single part" - this indicates that the value of the setting [`max_bytes_to_merge_at_max_space_in_pool`](https://clickhouse.com/docs/en/operations/settings/merge-tree-settings#max-bytes-to-merge-at-max-space-in-pool) will be ignored. For example, [`max_bytes_to_merge_at_max_space_in_pool`](https://clickhouse.com/docs/en/operations/settings/merge-tree-settings#max-bytes-to-merge-at-max-space-in-pool) is by default set to 150 GB. When running OPTIMIZE TABLE ... FINAL, the remaining single part could exceed even this size. This is another important consideration and reason not to generally use this command, since merging a large number of 150 GB parts into a single part could require a significant amount of time and/or memory.
1032

1133

0 commit comments

Comments
 (0)