Merge branch 'master' into async-metrics-sockets

Author: alexey-milovidov

.github/ISSUE_TEMPLATE/30_unexpected-behaviour.yaml

@@ -15,7 +15,12 @@ body:
   - type: textarea
     attributes:
       label: Describe the unexpected behaviour
-      description: A clear and concise description of what works not as it is supposed to.
+      description: A clear and concise description of what doesn't work as it is supposed to.
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: Which ClickHouse versions are affected?
     validations:
       required: true
   - type: textarea
@@ -26,7 +31,7 @@ body:
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true

.github/ISSUE_TEMPLATE/35_incomplete_implementation.yaml

@@ -26,7 +26,7 @@ body:
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true

.github/ISSUE_TEMPLATE/45_usability-issue.yaml

@@ -1,4 +1,4 @@
-name: Usability issue
+name: Usability improvement request
 description: Report something can be made more convenient to use
 labels: ["usability"]
 body:
@@ -14,28 +14,26 @@ body:
       required: false
   - type: textarea
     attributes:
-      label: Describe the unexpected behaviour
-      description: A clear and concise description of what works not as it is supposed to.
+      label: Describe the improvement
+      description: A clear and concise description of what you want to happen
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: Which ClickHouse versions are affected?
     validations:
       required: true
   - type: textarea
     attributes:
       label: How to reproduce
       description: |
-        * Which ClickHouse server version to use
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true
-  - type: textarea
-    attributes:
-      label: Expected behavior
-      description: A clear and concise description of what you expected to happen.
-    validations:
-      required: false
   - type: textarea
     attributes:
       label: Error message and/or stacktrace

.github/ISSUE_TEMPLATE/50_build-issue.yaml

@@ -12,33 +12,39 @@ body:
       description: Put your company name or project description here.
     validations:
       required: false
+  - type: textarea
+    attributes:
+      label: Describe the problem
+      description: A clear and concise description of what doesn't work as it is supposed to.
+    validations:
+      required: true
   - type: textarea
     attributes:
       label: Operating system
       description: OS kind or distribution, specific version/release, non-standard kernel if any. If you are trying to build inside virtual machine, please mention it too.
     validations:
-      required: true
+      required: false
   - type: textarea
     attributes:
       label: CMake version
       description: The output of `cmake --version`.
     validations:
-      required: true
+      required: false
   - type: textarea
     attributes:
       label: Ninja version
       description: The output of `ninja --version`.
     validations:
-      required: true
+      required: false
   - type: textarea
     attributes:
       label: Compiler name and version
       description: We recommend to use clang. The version can be obtained via `clang --version`.
     validations:
-      required: true
+      required: false
   - type: textarea
     attributes:
-      label: Full cmake and/or ninja output
-      description: Please include everything!
+      label: Full cmake and/or ninja output with the error
+      description: Please include everything (use https://pastila.nl/ for large output)!
     validations:
       required: true

.github/ISSUE_TEMPLATE/70_performance-issue.yaml

@@ -18,15 +18,19 @@ body:
       description: What exactly works slower than expected?
     validations:
       required: true
+  - type: textarea
+    attributes:
+      label: Which ClickHouse versions are affected?
+    validations:
+      required: true
   - type: textarea
     attributes:
       label: How to reproduce
       description: |
-        * Which ClickHouse server version to use
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true

.github/ISSUE_TEMPLATE/80_backward-compatibility.yaml

@@ -18,15 +18,19 @@ body:
       description: A clear and concise description of what works not as it is supposed to.
     validations:
       required: true
+  - type: textarea
+    attributes:
+      label: Which ClickHouse versions are affected?
+    validations:
+      required: true
   - type: textarea
     attributes:
       label: How to reproduce
       description: |
-        * Which ClickHouse server version to use
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true

.github/ISSUE_TEMPLATE/85_bug-report.yaml

@@ -52,7 +52,7 @@ body:
         * Which interface to use, if matters
         * Non-default settings, if any
         * `CREATE TABLE` statements for all tables involved
-        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+        * Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/c81bec37a58757be1e2b1ac6f20a62b3f14a31f1/programs/obfuscator/Obfuscator.cpp#L55-L95) if necessary
         * Queries to run that lead to unexpected result
     validations:
       required: true

ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt

@@ -1,4 +1,4 @@
-personal_ws-1.1 en 3238 
+personal_ws-1.1 en 3238
 AArch
 ACLs
 ALTERs
@@ -1850,7 +1850,6 @@ formatter
 formatters
 fqdn
 frac
-freezed
 frictionless
 fromDaysSinceYearZero
 fromModifiedJulianDay

docs/en/development/continuous-integration.md

@@ -143,7 +143,7 @@ mkdir build_tidy
 Runs [stateless functional tests](tests.md#functional-tests) for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc.
 Look at the report to see which tests fail, then reproduce the failure locally as described [here](/development/tests#functional-tests).
 Note that you have to use the correct build configuration to reproduce -- a test might fail under AddressSanitizer but pass in Debug.
-Download the binary from [CI build checks page](/install#install-a-ci-generated-binary), or build it locally.
+Download the binary from [CI build checks page](/install/advanced), or build it locally.
 
 ## Functional Stateful Tests {#functional-stateful-tests}
 

docs/en/engines/table-engines/mergetree-family/invertedindexes.md

@@ -54,28 +54,34 @@ CREATE TABLE tab
 (
     `key` UInt64,
     `str` String,
-    INDEX inv_idx(str) TYPE gin(0) GRANULARITY 1
+    INDEX inv_idx(str) TYPE gin(tokenizer = 'default|ngram|noop' [, ngram_size = N] [, max_rows_per_postings_list = M]) GRANULARITY 1
 )
 ENGINE = MergeTree
 ORDER BY key
 ```
 
-where `N` specifies the tokenizer:
+where `tokenizer` specifies the tokenizer:
 
-- `gin(0)` (or shorter: `gin()`) set the tokenizer to "tokens", i.e. split strings along spaces,
-- `gin(N)` with `N` between 2 and 8 sets the tokenizer to "ngrams(N)"
+- `default` set the tokenizer to "tokens('default')", i.e. split strings along non-alphanumeric characters.
+- `ngram` set the tokenizer to "tokens('ngram')". i.e. splits strings to equal size terms.
+- `noop` set the tokenizer to "tokens('noop')", i.e. every value itself is a term.
 
-The maximum rows per postings list can be specified as the second parameter. This parameter can be used to control postings list sizes to avoid generating huge postings list files. The following variants exist:
+The ngram size can be specified via the `ngram_size` parameter. This is an optional parameter. The following variants exist:
 
-- `gin(ngrams, max_rows_per_postings_list)`: Use given max_rows_per_postings_list (assuming it is not 0)
-- `gin(ngrams, 0)`: No limitation of maximum rows per postings list
-- `gin(ngrams)`: Use a default maximum rows which is 64K.
+- `ngram_size = N`: with `N` between 2 and 8 sets the tokenizer to "tokens('ngram', N)".
+- If not specified: Use a default ngram size which is 3.
+
+The maximum rows per postings list can be specified via an optional `max_rows_per_postings_list`. This parameter can be used to control postings list sizes to avoid generating huge postings list files. The following variants exist:
+
+- `max_rows_per_postings_list = 0`: No limitation of maximum rows per postings list.
+- `max_rows_per_postings_list = M`: with `M` should be at least 8192.
+- If not specified: Use a default maximum rows which is 64K.
 
 Being a type of skipping index, full-text indexes can be dropped or added to a column after table creation:
 
 ```sql
 ALTER TABLE tab DROP INDEX inv_idx;
-ALTER TABLE tab ADD INDEX inv_idx(s) TYPE gin(2);
+ALTER TABLE tab ADD INDEX inv_idx(s) TYPE gin(tokenizer = 'default');
 ```
 
 To use the index, no special functions or syntax are required. Typical string search predicates automatically leverage the index. As