Merge branch 'main' into image_component

Author: Blargian

contribute/style-guide.md

@@ -92,6 +92,38 @@ Code blocks:
 - Have a title (optional) such as 'Query' or 'Response'
 - Use language `response` if it is for the result of a query.
 
+### Highlighting
+
+You can highlight lines in a code block using the following keywords:
+
+- `highlight-next-line` 
+- `highlight-start`
+- `highlight-end`
+
+These keywords should be added as comments in the codeblock with the appropriate
+escape symbol for the codeblock language. 
+
+For example, if the codeblock is SQL:
+
+```text
+SELECT UserID, count(UserID) AS Count
+-- highlight-next-line
+FROM mv_hits_URL_UserID
+WHERE URL = 'http://public_search'
+GROUP BY UserID
+ORDER BY Count DESC
+LIMIT 10;
+```
+
+If the codeblock is a response: 
+
+```text
+10 rows in set. Elapsed: 0.026 sec.
+# highlight-next-line
+Processed 335.87 thousand rows,
+13.54 MB (12.91 million rows/s., 520.38 MB/s.)
+```
+
 ### Associated markdown rule or CI check
 
 - [`MD040` enforces that codeblocks have a language specified](/scripts/.markdownlint-cli2.yaml)

docs/cloud/bestpractices/avoidnullablecolumns.md

@@ -13,7 +13,7 @@ To avoid `Nullable` columns, consider setting a default value for that column.
 CREATE TABLE default.sample
 (
     `x` Int8,
-    # highlight-next-line
+    -- highlight-next-line
     `y` Nullable(Int8)
 )
 ENGINE = MergeTree
@@ -25,7 +25,7 @@ use
 CREATE TABLE default.sample2
 (
     `x` Int8,
-    # highlight-next-line
+    -- highlight-next-line
     `y` Int8 DEFAULT 0
 )
 ENGINE = MergeTree

docs/data-modeling/backfilling.md

@@ -465,14 +465,14 @@ Notice our memory usage here is `639.47 MiB`.
 
 ##### Tuning performance & resources {#tuning-performance--resources}
 
-Several factors will determine the performance and resources used in the above scenario. We recommend readers understand insert mechanics documented in detail [here](/integrations/s3/performance#using-threads-for-reads) prior to attempting to tune. In summary:
+Several factors will determine the performance and resources used in the above scenario. Before attempting to tune, we recommend readers understand the insert mechanics documented in detail in the [Using Threads for Reads](/integrations/s3/performance#using-threads-for-reads) section of the [Optimizing for S3 Insert and Read Performance guide](/integrations/s3/performance). In summary:
 
 - **Read Parallelism** - The number of threads used to read. Controlled through [`max_threads`](/operations/settings/settings#max_threads). In ClickHouse Cloud this is determined by the instance size with it defaulting to the number of vCPUs. Increasing this value may improve read performance at the expense of greater memory usage.
 - **Insert Parallelism** - The number of insert threads used to insert. Controlled through [`max_insert_threads`](/operations/settings/settings#max_insert_threads). In ClickHouse Cloud this is determined by the instance size (between 2 and 4) and is set to 1 in OSS. Increasing this value may improve performance at the expense of greater memory usage.
 - **Insert Block Size** -  data is processed in a loop where it is pulled, parsed, and formed into in-memory insert blocks based on the [partitioning key](/engines/table-engines/mergetree-family/custom-partitioning-key). These blocks are sorted, optimized, compressed, and written to storage as new [data parts](/parts). The size of the insert block, controlled by settings [`min_insert_block_size_rows`](/operations/settings/settings#min_insert_block_size_rows) and [`min_insert_block_size_bytes`](/operations/settings/settings#min_insert_block_size_bytes) (uncompressed), impacts memory usage and disk I/O. Larger blocks use more memory but create fewer parts, reducing I/O and background merges. These settings represent minimum thresholds (whichever is reached first triggers a flush).
 - **Materialized view block size** - As well as the above mechanics for the main insert, prior to insertion into materialized views, blocks are also squashed for more efficient processing. The size of these blocks is determined by the settings [`min_insert_block_size_bytes_for_materialized_views`](/operations/settings/settings#min_insert_block_size_bytes_for_materialized_views) and [`min_insert_block_size_rows_for_materialized_views`](/operations/settings/settings#min_insert_block_size_rows_for_materialized_views). Larger blocks allow more efficient processing at the expense of greater memory usage. By default, these settings revert to the values of the source table settings [`min_insert_block_size_rows`](/operations/settings/settings#min_insert_block_size_rows) and [`min_insert_block_size_bytes`](/operations/settings/settings#min_insert_block_size_bytes), respectively.
 
-For improving performance, users can follow the guidelines outlined [here](/integrations/s3/performance#tuning-threads-and-block-size-for-inserts). It should not be necessary to also modify `min_insert_block_size_bytes_for_materialized_views` and `min_insert_block_size_rows_for_materialized_views` to improve performance in most cases. If these are modified, use the same best practices as discussed for `min_insert_block_size_rows` and `min_insert_block_size_bytes`.
+For improving performance, users can follow the guidelines outlined in the [Tuning Threads and Block Size for Inserts](/integrations/s3/performance#tuning-threads-and-block-size-for-inserts) section of the [Optimizing for S3 Insert and Read Performance guide](/integrations/s3/performance). It should not be necessary to also modify `min_insert_block_size_bytes_for_materialized_views` and `min_insert_block_size_rows_for_materialized_views` to improve performance in most cases. If these are modified, use the same best practices as discussed for `min_insert_block_size_rows` and `min_insert_block_size_bytes`.
 
 To minimize memory, users may wish to experiment with these settings. This will invariably lower performance. Using the earlier query, we show examples below.
 

docs/guides/best-practices/sparse-primary-indexes.md

@@ -147,7 +147,7 @@ The response is:
 └────────────────────────────────┴───────┘
 
 10 rows in set. Elapsed: 0.022 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 8.87 million rows,
 70.45 MB (398.53 million rows/s., 3.17 GB/s.)
 ```
@@ -184,7 +184,7 @@ CREATE TABLE hits_UserID_URL
     `EventTime` DateTime
 )
 ENGINE = MergeTree
-// highlight-next-line
+-- highlight-next-line
 PRIMARY KEY (UserID, URL)
 ORDER BY (UserID, URL, EventTime)
 SETTINGS index_granularity = 8192, index_granularity_bytes = 0, compress_primary_key = 0;
@@ -519,7 +519,7 @@ The response is:
 └────────────────────────────────┴───────┘
 
 10 rows in set. Elapsed: 0.005 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 8.19 thousand rows,
 740.18 KB (1.53 million rows/s., 138.59 MB/s.)
 ```
@@ -530,13 +530,13 @@ The output for the ClickHouse client is now showing that instead of doing a full
 If <a href="https://clickhouse.com/docs/operations/server-configuration-parameters/settings/#server_configuration_parameters-logger" target="_blank">trace logging</a> is enabled then the ClickHouse server log file shows that ClickHouse was running a <a href="https://github.com/ClickHouse/ClickHouse/blob/22.3/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp#L1452" target="_blank">binary search</a> over the 1083 UserID index marks, in order to identify granules that possibly can contain rows with a UserID column value of `749927693`. This requires 19 steps with an average time complexity of `O(log2 n)`:
 ```response
 ...Executor): Key condition: (column 0 in [749927693, 749927693])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Running binary search on index range for part all_1_9_2 (1083 marks)
 ...Executor): Found (LEFT) boundary mark: 176
 ...Executor): Found (RIGHT) boundary mark: 177
 ...Executor): Found continuous range in 19 steps
 ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               1/1083 marks by primary key, 1 marks to read from 1 ranges
 ...Reading ...approx. 8192 rows starting from 1441792
 ```
@@ -584,7 +584,7 @@ The response looks like:
 │                       UserID                                                          │
 │                     Condition: (UserID in [749927693, 749927693])                     │
 │                     Parts: 1/1                                                        │
-// highlight-next-line
+# highlight-next-line
 │                     Granules: 1/1083                                                  │
 └───────────────────────────────────────────────────────────────────────────────────────┘
 
@@ -753,7 +753,7 @@ The response is: <a name="query-on-url-slow"></a>
 └────────────┴───────┘
 
 10 rows in set. Elapsed: 0.086 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 8.81 million rows,
 799.69 MB (102.11 million rows/s., 9.27 GB/s.)
 ```
@@ -764,11 +764,11 @@ If [trace_logging](/operations/server-configuration-parameters/settings#logger)
 ```response
 ...Executor): Key condition: (column 1 in ['http://public_search',
                                            'http://public_search'])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Used generic exclusion search over index for part all_1_9_2
               with 1537 steps
 ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               1076/1083 marks by primary key, 1076 marks to read from 5 ranges
 ...Executor): Reading approx. 8814592 rows with 10 streams
 ```
@@ -917,7 +917,7 @@ CREATE TABLE hits_URL_UserID
     `EventTime` DateTime
 )
 ENGINE = MergeTree
-// highlight-next-line
+-- highlight-next-line
 PRIMARY KEY (URL, UserID)
 ORDER BY (URL, UserID, EventTime)
 SETTINGS index_granularity = 8192, index_granularity_bytes = 0, compress_primary_key = 0;
@@ -954,7 +954,7 @@ This is the resulting primary key:
 That can now be used to significantly speed up the execution of our example query filtering on the URL column in order to calculate the top 10 users that most frequently clicked on the URL "http://public_search":
 ```sql
 SELECT UserID, count(UserID) AS Count
-// highlight-next-line
+-- highlight-next-line
 FROM hits_URL_UserID
 WHERE URL = 'http://public_search'
 GROUP BY UserID
@@ -980,7 +980,7 @@ The response is:
 └────────────┴───────┘
 
 10 rows in set. Elapsed: 0.017 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 319.49 thousand rows,
 11.38 MB (18.41 million rows/s., 655.75 MB/s.)
 ```
@@ -994,13 +994,13 @@ The corresponding trace log in the ClickHouse server log file confirms that:
 ```response
 ...Executor): Key condition: (column 0 in ['http://public_search',
                                            'http://public_search'])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Running binary search on index range for part all_1_9_2 (1083 marks)
 ...Executor): Found (LEFT) boundary mark: 644
 ...Executor): Found (RIGHT) boundary mark: 683
 ...Executor): Found continuous range in 19 steps
 ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               39/1083 marks by primary key, 39 marks to read from 1 ranges
 ...Executor): Reading approx. 319488 rows with 2 streams
 ```
@@ -1045,19 +1045,19 @@ The response is:
 └────────────────────────────────┴───────┘
 
 10 rows in set. Elapsed: 0.024 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 8.02 million rows,
 73.04 MB (340.26 million rows/s., 3.10 GB/s.)
 ```
 
 Server Log:
 ```response
 ...Executor): Key condition: (column 1 in [749927693, 749927693])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Used generic exclusion search over index for part all_1_9_2
               with 1453 steps
 ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               980/1083 marks by primary key, 980 marks to read from 23 ranges
 ...Executor): Reading approx. 8028160 rows with 10 streams
 ```
@@ -1108,7 +1108,7 @@ ClickHouse is storing the [column data files](#data-is-stored-on-disk-ordered-by
 The implicitly created table (and its primary index) backing the materialized view can now be used to significantly speed up the execution of our example query filtering on the URL column:
 ```sql
 SELECT UserID, count(UserID) AS Count
-// highlight-next-line
+-- highlight-next-line
 FROM mv_hits_URL_UserID
 WHERE URL = 'http://public_search'
 GROUP BY UserID
@@ -1133,7 +1133,7 @@ The response is:
 └────────────┴───────┘
 
 10 rows in set. Elapsed: 0.026 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 335.87 thousand rows,
 13.54 MB (12.91 million rows/s., 520.38 MB/s.)
 ```
@@ -1145,11 +1145,11 @@ The corresponding trace log in the ClickHouse server log file confirms that Clic
 ```response
 ...Executor): Key condition: (column 0 in ['http://public_search',
                                            'http://public_search'])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Running binary search on index range ...
 ...
 ...Executor): Selected 4/4 parts by partition key, 4 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               41/1083 marks by primary key, 41 marks to read from 4 ranges
 ...Executor): Reading approx. 335872 rows with 4 streams
 ```
@@ -1193,7 +1193,7 @@ ClickHouse is storing the [column data files](#data-is-stored-on-disk-ordered-by
 The hidden table (and its primary index) created by the projection can now be (implicitly) used to significantly speed up the execution of our example query filtering on the URL column. Note that the query is syntactically targeting the source table of the projection.
 ```sql
 SELECT UserID, count(UserID) AS Count
-// highlight-next-line
+-- highlight-next-line
 FROM hits_UserID_URL
 WHERE URL = 'http://public_search'
 GROUP BY UserID
@@ -1218,7 +1218,7 @@ The response is:
 └────────────┴───────┘
 
 10 rows in set. Elapsed: 0.029 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 319.49 thousand rows, 1
 1.38 MB (11.05 million rows/s., 393.58 MB/s.)
 ```
@@ -1231,14 +1231,14 @@ The corresponding trace log in the ClickHouse server log file confirms that Clic
 ```response
 ...Executor): Key condition: (column 0 in ['http://public_search',
                                            'http://public_search'])
-// highlight-next-line
+# highlight-next-line
 ...Executor): Running binary search on index range for part prj_url_userid (1083 marks)
 ...Executor): ...
-// highlight-next-line
+# highlight-next-line
 ...Executor): Choose complete Normal projection prj_url_userid
 ...Executor): projection required columns: URL, UserID
 ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key,
-// highlight-next-line
+# highlight-next-line
               39/1083 marks by primary key, 39 marks to read from 1 ranges
 ...Executor): Reading approx. 319488 rows with 2 streams
 ```
@@ -1317,7 +1317,7 @@ CREATE TABLE hits_URL_UserID_IsRobot
     `IsRobot` UInt8
 )
 ENGINE = MergeTree
-// highlight-next-line
+-- highlight-next-line
 PRIMARY KEY (URL, UserID, IsRobot);
 ```
 
@@ -1345,7 +1345,7 @@ CREATE TABLE hits_IsRobot_UserID_URL
     `IsRobot` UInt8
 )
 ENGINE = MergeTree
-// highlight-next-line
+-- highlight-next-line
 PRIMARY KEY (IsRobot, UserID, URL);
 ```
 And populate it with the same 8.87 million rows that we used to populate the previous table:
@@ -1385,7 +1385,7 @@ The response is:
 └─────────┘
 
 1 row in set. Elapsed: 0.026 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 7.92 million rows,
 31.67 MB (306.90 million rows/s., 1.23 GB/s.)
 ```
@@ -1403,7 +1403,7 @@ The response is:
 └─────────┘
 
 1 row in set. Elapsed: 0.003 sec.
-// highlight-next-line
+# highlight-next-line
 Processed 20.32 thousand rows,
 81.28 KB (6.61 million rows/s., 26.44 MB/s.)
 ```

docs/guides/sre/user-management/ssl-user-auth.md

@@ -14,10 +14,13 @@ import SelfManaged from '@site/docs/_snippets/_self_managed_only_no_roadmap.md';
 This guide provides simple and minimal settings to configure authentication with SSL user certificates. The tutorial builds on the [Configuring SSL-TLS user guide](../configuring-ssl.md).
 
 :::note
-SSL user authentication is supported when using the `https` or native interfaces only.
-It is not currently used in gRPC or PostgreSQL/MySQL emulation ports.
+SSL user authentication is supported when using the `https`, `native`, `mysql`, and `postgresql` interfaces.
 
 ClickHouse nodes need `<verificationMode>strict</verificationMode>` set for secure authentication (although `relaxed` will work for testing purposes).
+
+If you use AWS NLB with the MySQL interface, you have to ask AWS support to enable the undocumented option:
+
+> I would like to be able to configure our NLB proxy protocol v2 as below `proxy_protocol_v2.client_to_server.header_placement,Value=on_first_ack`.
 :::
 
 ## 1. Create SSL user certificates {#1-create-ssl-user-certificates}

docs/integrations/data-ingestion/gcs/index.md

@@ -156,7 +156,7 @@ CREATE TABLE trips_gcs
 ENGINE = MergeTree
 PARTITION BY toYYYYMM(pickup_date)
 ORDER BY pickup_datetime
-# highlight-next-line
+-- highlight-next-line
 SETTINGS storage_policy='gcs_main'
 ```
 
@@ -469,10 +469,10 @@ zk_synced_followers	2
 # highlight-end
 ```
 
-
 ### Start ClickHouse server {#start-clickhouse-server}
 
 On `chnode1` and `chnode` run:
+
 ```bash
 sudo service clickhouse-server start
 ```
@@ -547,7 +547,7 @@ cache_path:
 ```
 #### Verify that tables created on the cluster are created on both nodes {#verify-that-tables-created-on-the-cluster-are-created-on-both-nodes}
 ```sql
-# highlight-next-line
+-- highlight-next-line
 create table trips on cluster 'cluster_1S_2R' (
  `trip_id` UInt32,
  `pickup_date` Date,
@@ -565,7 +565,7 @@ create table trips on cluster 'cluster_1S_2R' (
 ENGINE = ReplicatedMergeTree
 PARTITION BY toYYYYMM(pickup_date)
 ORDER BY pickup_datetime
-# highlight-next-line
+-- highlight-next-line
 SETTINGS storage_policy='gcs_main'
 ```
 ```response