From 5bebdc8c645c5d8a277c6b25fbcd615e8e8d23e8 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Thu, 27 Nov 2025 18:53:31 +0800 Subject: [PATCH 01/25] Update the setting defaults in sql plugin Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 64785196a4..d55f5202e1 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -72,18 +72,30 @@ Requests to the `_plugins/_ppl` and `_plugins/_sql` endpoints include index name Setting | Default | Description :--- | :--- | :--- -`plugins.sql.enabled` | True | Change to `false` to disable the `SQL` support in the plugin. -`plugins.ppl.enabled` | True | Change to `false` to disable the `PPL` support in the plugin. -`plugins.sql.slowlog` | 2 seconds | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. -`plugins.sql.cursor.keep_alive` | 1 minute | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. +`plugins.sql.enabled` | true | Change to `false` to disable the `SQL` support in the plugin. +`plugins.ppl.enabled` | true | Change to `false` to disable the `PPL` support in the plugin. +`plugins.sql.slowlog` | 2 | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. +`plugins.sql.cursor.keep_alive` | 1m | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. `plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine. -`plugins.query.size_limit` | 10000 | Sets the default size of index that the query engine fetches from OpenSearch. +`plugins.query.size_limit` | 10000 | Sets the maximum amount of rows returned from a query execution results. `plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin. `plugins.query.field_type_tolerance` | true | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. +`plugins.query.buckets` | 10000 | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. `plugins.calcite.enabled` | true | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearches, joins, and lookup operations. +`plugins.calcite.pushdown.enabled` | true | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. +`plugins.calcite.fallback.allowed` | false | Change to `true` to allow fallback to v2 engine. +`plugins.calcite.pushdown.rowcount.estimation.factor` | 0.9 | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. `plugins.calcite.all_join_types.allowed` | false | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. `plugins.ppl.syntax.legacy.preferred` | true | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. `plugins.ppl.values.max.limit` | 0 | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. +`plugins.ppl.rex.max_match.limit` | 10 | Maximum number of matches to extract in `rex` command. +`plugins.ppl.subsearch.maxout` | 10000 | Sets the maximum of rows to return from subsearch. +`plugins.ppl.join.subsearch_maxout` | 50000 | Sets the maximum of rows from subsearch to join against. +`plugins.ppl.pattern.method` | "simple_pattern" | Sets the default patterns method. Another supported method is "brain". +`plugins.ppl.pattern.mode` | "label" | Sets the default patterns mode. Another supported mode is "aggregation". +`plugins.ppl.pattern.max.sample.count` | 10 | Sets the default max sample logs returned per pattern in aggregation mode. +`plugins.ppl.pattern.buffer.limit` | 100000 | Sets the default size of internal temporary buffer used in "brain" algorithm. +`plugins.ppl.pattern.show.numbered.token` | false | Change to `true` to turn on numbered token output format. ## Spark connector settings From 97697be57869a0362b7f3f5539992caa5c640c4b Mon Sep 17 00:00:00 2001 From: Fanit Kolchina Date: Tue, 2 Dec 2025 15:44:12 -0500 Subject: [PATCH 02/25] Doc review Signed-off-by: Fanit Kolchina Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 54 +++++++++++++++++---------------- 1 file changed, 28 insertions(+), 26 deletions(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index d55f5202e1..fbcb69faa3 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -70,32 +70,34 @@ Requests to the `_plugins/_ppl` and `_plugins/_sql` endpoints include index name ## Available settings -Setting | Default | Description -:--- | :--- | :--- -`plugins.sql.enabled` | true | Change to `false` to disable the `SQL` support in the plugin. -`plugins.ppl.enabled` | true | Change to `false` to disable the `PPL` support in the plugin. -`plugins.sql.slowlog` | 2 | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. -`plugins.sql.cursor.keep_alive` | 1m | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. -`plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine. -`plugins.query.size_limit` | 10000 | Sets the maximum amount of rows returned from a query execution results. -`plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin. -`plugins.query.field_type_tolerance` | true | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. -`plugins.query.buckets` | 10000 | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. -`plugins.calcite.enabled` | true | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearches, joins, and lookup operations. -`plugins.calcite.pushdown.enabled` | true | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. -`plugins.calcite.fallback.allowed` | false | Change to `true` to allow fallback to v2 engine. -`plugins.calcite.pushdown.rowcount.estimation.factor` | 0.9 | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. -`plugins.calcite.all_join_types.allowed` | false | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. -`plugins.ppl.syntax.legacy.preferred` | true | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. -`plugins.ppl.values.max.limit` | 0 | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. -`plugins.ppl.rex.max_match.limit` | 10 | Maximum number of matches to extract in `rex` command. -`plugins.ppl.subsearch.maxout` | 10000 | Sets the maximum of rows to return from subsearch. -`plugins.ppl.join.subsearch_maxout` | 50000 | Sets the maximum of rows from subsearch to join against. -`plugins.ppl.pattern.method` | "simple_pattern" | Sets the default patterns method. Another supported method is "brain". -`plugins.ppl.pattern.mode` | "label" | Sets the default patterns mode. Another supported mode is "aggregation". -`plugins.ppl.pattern.max.sample.count` | 10 | Sets the default max sample logs returned per pattern in aggregation mode. -`plugins.ppl.pattern.buffer.limit` | 100000 | Sets the default size of internal temporary buffer used in "brain" algorithm. -`plugins.ppl.pattern.show.numbered.token` | false | Change to `true` to turn on numbered token output format. +The following table lists the available SQL and PPL settings. + +| Setting | Default | Description | +| :---------------------------------------------------- | :----------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `plugins.sql.enabled` | `true` | Change to `false` to disable SQL support in the plugin. | +| `plugins.ppl.enabled` | `true` | Change to `false` to disable PPL support in the plugin. | +| `plugins.sql.slowlog` | `2` | Sets the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. | +| `plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. | +| `plugins.query.memory_limit` | `85%` | Sets the heap memory usage limit for the query engine’s circuit breaker. | +| `plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query. | +| `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. | +| `plugins.query.field_type_tolerance` | `true` | If set to `false`, an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` reduces to `1`. If `true`, the array is preserved. | +| `plugins.query.buckets` | `10000` | Sets the number of aggregation buckets returned in a single response. The default value matches `plugins.query.size_limit`. | +| `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL features such as subsearches, joins, and lookup operations. | +| `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. | +| `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. | +| `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | Sets a factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. | +| `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types (such as `RIGHT`, `FULL`, and `CROSS`). Change to `true` to allow these join operations. | +| `plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When set to `false`, uses newer syntax standards. | +| `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. | +| `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. | +| `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum number of rows returned from a subsearch. | +| `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum number of rows from a subsearch used in a join. | +| `plugins.ppl.pattern.method` | `simple_pattern` | Sets the default algorithm used by the PPL patterns command to detect patterns in log data. Valid values are `simple_pattern` (a lightweight, fast pattern-extraction method) and `brain` (a more advanced algorithm for deeper pattern analysis). | +| `plugins.ppl.pattern.mode` | `label` | Sets the default pattern-matching mode used by the PPL patterns command. This determines how log patterns are generated and returned. Valid values are `label` (produces concise, high-level pattern labels) and `aggregation` (returns detailed pattern groups with sample logs and counts). | +| `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. | +| `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. | +| `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered-token output format. | ## Spark connector settings From fcce1ddb5f6a209dbc5b8856664a2ae8b3d08b49 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Thu, 4 Dec 2025 10:57:38 +0800 Subject: [PATCH 03/25] Revert "Doc review" This reverts commit b7fa236dbf95187e97c9abef8e86450ee0335ee8. Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 54 ++++++++++++++++----------------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index fbcb69faa3..d55f5202e1 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -70,34 +70,32 @@ Requests to the `_plugins/_ppl` and `_plugins/_sql` endpoints include index name ## Available settings -The following table lists the available SQL and PPL settings. - -| Setting | Default | Description | -| :---------------------------------------------------- | :----------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `plugins.sql.enabled` | `true` | Change to `false` to disable SQL support in the plugin. | -| `plugins.ppl.enabled` | `true` | Change to `false` to disable PPL support in the plugin. | -| `plugins.sql.slowlog` | `2` | Sets the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. | -| `plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. | -| `plugins.query.memory_limit` | `85%` | Sets the heap memory usage limit for the query engine’s circuit breaker. | -| `plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query. | -| `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. | -| `plugins.query.field_type_tolerance` | `true` | If set to `false`, an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` reduces to `1`. If `true`, the array is preserved. | -| `plugins.query.buckets` | `10000` | Sets the number of aggregation buckets returned in a single response. The default value matches `plugins.query.size_limit`. | -| `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL features such as subsearches, joins, and lookup operations. | -| `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. | -| `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. | -| `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | Sets a factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. | -| `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types (such as `RIGHT`, `FULL`, and `CROSS`). Change to `true` to allow these join operations. | -| `plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When set to `false`, uses newer syntax standards. | -| `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. | -| `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. | -| `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum number of rows returned from a subsearch. | -| `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum number of rows from a subsearch used in a join. | -| `plugins.ppl.pattern.method` | `simple_pattern` | Sets the default algorithm used by the PPL patterns command to detect patterns in log data. Valid values are `simple_pattern` (a lightweight, fast pattern-extraction method) and `brain` (a more advanced algorithm for deeper pattern analysis). | -| `plugins.ppl.pattern.mode` | `label` | Sets the default pattern-matching mode used by the PPL patterns command. This determines how log patterns are generated and returned. Valid values are `label` (produces concise, high-level pattern labels) and `aggregation` (returns detailed pattern groups with sample logs and counts). | -| `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. | -| `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. | -| `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered-token output format. | +Setting | Default | Description +:--- | :--- | :--- +`plugins.sql.enabled` | true | Change to `false` to disable the `SQL` support in the plugin. +`plugins.ppl.enabled` | true | Change to `false` to disable the `PPL` support in the plugin. +`plugins.sql.slowlog` | 2 | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. +`plugins.sql.cursor.keep_alive` | 1m | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. +`plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine. +`plugins.query.size_limit` | 10000 | Sets the maximum amount of rows returned from a query execution results. +`plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin. +`plugins.query.field_type_tolerance` | true | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. +`plugins.query.buckets` | 10000 | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. +`plugins.calcite.enabled` | true | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearches, joins, and lookup operations. +`plugins.calcite.pushdown.enabled` | true | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. +`plugins.calcite.fallback.allowed` | false | Change to `true` to allow fallback to v2 engine. +`plugins.calcite.pushdown.rowcount.estimation.factor` | 0.9 | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. +`plugins.calcite.all_join_types.allowed` | false | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. +`plugins.ppl.syntax.legacy.preferred` | true | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. +`plugins.ppl.values.max.limit` | 0 | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. +`plugins.ppl.rex.max_match.limit` | 10 | Maximum number of matches to extract in `rex` command. +`plugins.ppl.subsearch.maxout` | 10000 | Sets the maximum of rows to return from subsearch. +`plugins.ppl.join.subsearch_maxout` | 50000 | Sets the maximum of rows from subsearch to join against. +`plugins.ppl.pattern.method` | "simple_pattern" | Sets the default patterns method. Another supported method is "brain". +`plugins.ppl.pattern.mode` | "label" | Sets the default patterns mode. Another supported mode is "aggregation". +`plugins.ppl.pattern.max.sample.count` | 10 | Sets the default max sample logs returned per pattern in aggregation mode. +`plugins.ppl.pattern.buffer.limit` | 100000 | Sets the default size of internal temporary buffer used in "brain" algorithm. +`plugins.ppl.pattern.show.numbered.token` | false | Change to `true` to turn on numbered token output format. ## Spark connector settings From 45c403b2f5835b2b3ee7adda8d212e5f49a09067 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Thu, 4 Dec 2025 11:04:10 +0800 Subject: [PATCH 04/25] add a new config plugins.ppl.query.timeout Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 50 +++++++++++++++++---------------- 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index d55f5202e1..b0a672ba12 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -72,30 +72,32 @@ Requests to the `_plugins/_ppl` and `_plugins/_sql` endpoints include index name Setting | Default | Description :--- | :--- | :--- -`plugins.sql.enabled` | true | Change to `false` to disable the `SQL` support in the plugin. -`plugins.ppl.enabled` | true | Change to `false` to disable the `PPL` support in the plugin. -`plugins.sql.slowlog` | 2 | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. -`plugins.sql.cursor.keep_alive` | 1m | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. -`plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine. -`plugins.query.size_limit` | 10000 | Sets the maximum amount of rows returned from a query execution results. -`plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin. -`plugins.query.field_type_tolerance` | true | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. -`plugins.query.buckets` | 10000 | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. -`plugins.calcite.enabled` | true | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearches, joins, and lookup operations. -`plugins.calcite.pushdown.enabled` | true | Change to `false` to disable the operator pushdown optimization. We recommend using the default value. -`plugins.calcite.fallback.allowed` | false | Change to `true` to allow fallback to v2 engine. -`plugins.calcite.pushdown.rowcount.estimation.factor` | 0.9 | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. -`plugins.calcite.all_join_types.allowed` | false | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. -`plugins.ppl.syntax.legacy.preferred` | true | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. -`plugins.ppl.values.max.limit` | 0 | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. -`plugins.ppl.rex.max_match.limit` | 10 | Maximum number of matches to extract in `rex` command. -`plugins.ppl.subsearch.maxout` | 10000 | Sets the maximum of rows to return from subsearch. -`plugins.ppl.join.subsearch_maxout` | 50000 | Sets the maximum of rows from subsearch to join against. -`plugins.ppl.pattern.method` | "simple_pattern" | Sets the default patterns method. Another supported method is "brain". -`plugins.ppl.pattern.mode` | "label" | Sets the default patterns mode. Another supported mode is "aggregation". -`plugins.ppl.pattern.max.sample.count` | 10 | Sets the default max sample logs returned per pattern in aggregation mode. -`plugins.ppl.pattern.buffer.limit` | 100000 | Sets the default size of internal temporary buffer used in "brain" algorithm. -`plugins.ppl.pattern.show.numbered.token` | false | Change to `true` to turn on numbered token output format. +`plugins.sql.enabled` | `true` | Change to `false` to disable the `SQL` support in the plugin. +`plugins.ppl.enabled` | `true` | Change to `false` to disable the `PPL` support in the plugin. +`plugins.sql.slowlog` | `2` | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. +`plugins.sql.cursor.keep_alive` | `1m` | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. +`plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the circuit breaker of the query engine. +`plugins.query.size_limit` | `10000` | Sets the maximum amount of rows returned from a query execution results. +`plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. +`plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. +`plugins.query.buckets` | `10000` | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. +`plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. +`plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. +`plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to v2 engine. +`plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. +`plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. +`plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. +`plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. +`plugins.ppl.rex.max_match.limit` | `10` | Maximum number of matches to extract in `rex` command. +`plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. +`plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. +`plugins.ppl.pattern.method` | `"simple_pattern"` | Sets the default patterns method. Another supported method is "brain". +`plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". +`plugins.ppl.pattern.max.sample.count` | `10` | Sets the default max sample logs returned per pattern in aggregation mode. +`plugins.ppl.pattern.buffer.limit` | `100000` | Sets the default size of internal temporary buffer used in "brain" algorithm. +`plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. +`plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. +`plugins.ppl.query.timeout` | `5m` | Configures the maximum execution time for PPL queries. When a query exceeds this timeout, it will be interrupted and return a timeout error. ## Spark connector settings From 06e9975840ffbf5060254d543a8d4a400e06c4e2 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:42:49 +0800 Subject: [PATCH 05/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index b0a672ba12..4fcbe6a6de 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -75,7 +75,7 @@ Setting | Default | Description `plugins.sql.enabled` | `true` | Change to `false` to disable the `SQL` support in the plugin. `plugins.ppl.enabled` | `true` | Change to `false` to disable the `PPL` support in the plugin. `plugins.sql.slowlog` | `2` | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. -`plugins.sql.cursor.keep_alive` | `1m` | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. +`plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. `plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the circuit breaker of the query engine. `plugins.query.size_limit` | `10000` | Sets the maximum amount of rows returned from a query execution results. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. From 222ae4274b3e669b9208d7acdc73d9c1373d6b27 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:43:21 +0800 Subject: [PATCH 06/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 4fcbe6a6de..deecb3996d 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -76,7 +76,7 @@ Setting | Default | Description `plugins.ppl.enabled` | `true` | Change to `false` to disable the `PPL` support in the plugin. `plugins.sql.slowlog` | `2` | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. `plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. -`plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the circuit breaker of the query engine. +`plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the query engine's circuit breaker. `plugins.query.size_limit` | `10000` | Sets the maximum amount of rows returned from a query execution results. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. `plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. From 49a0709f5252f2813b1f82d308f8c2198d712c00 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:43:43 +0800 Subject: [PATCH 07/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index deecb3996d..762af4794b 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -77,7 +77,7 @@ Setting | Default | Description `plugins.sql.slowlog` | `2` | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. `plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. `plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the query engine's circuit breaker. -`plugins.query.size_limit` | `10000` | Sets the maximum amount of rows returned from a query execution results. +`plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query execution. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. `plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. `plugins.query.buckets` | `10000` | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. From 52de914cd796c6c2253993d28aa265996893bad9 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:43:58 +0800 Subject: [PATCH 08/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 762af4794b..50c1468ca9 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -79,7 +79,7 @@ Setting | Default | Description `plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the query engine's circuit breaker. `plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query execution. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. -`plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` will be reduced to `1`. If `true`, then the array is preserved. Default is `true`. +`plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` is reduced to `1`. If `true`, then the array is preserved. Default is `true`. `plugins.query.buckets` | `10000` | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. From e9957abb3375e0374f0b0ada85d033d8208412a8 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:44:16 +0800 Subject: [PATCH 09/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 50c1468ca9..3ccd299f5b 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -80,7 +80,7 @@ Setting | Default | Description `plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query execution. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. `plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` is reduced to `1`. If `true`, then the array is preserved. Default is `true`. -`plugins.query.buckets` | `10000` | Sets how many aggregation buckets will return in a single response. The default value equals to `plugins.query.size_limit`. +`plugins.query.buckets` | `10000` | Sets the number of aggregation buckets returned in a single response. Default is the `plugins.query.size_limit` value. `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to v2 engine. From 3693b65e58bc11b137862037e42b731a3c13b085 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:44:24 +0800 Subject: [PATCH 10/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 3ccd299f5b..85ad03e1c4 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -93,7 +93,7 @@ Setting | Default | Description `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. `plugins.ppl.pattern.method` | `"simple_pattern"` | Sets the default patterns method. Another supported method is "brain". `plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". -`plugins.ppl.pattern.max.sample.count` | `10` | Sets the default max sample logs returned per pattern in aggregation mode. +`plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the default size of internal temporary buffer used in "brain" algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. From 1e3aa438439cf783f0065ba32fd6c4ecf4e72a11 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:44:37 +0800 Subject: [PATCH 11/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 85ad03e1c4..034dd743ab 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -83,7 +83,7 @@ Setting | Default | Description `plugins.query.buckets` | `10000` | Sets the number of aggregation buckets returned in a single response. Default is the `plugins.query.size_limit` value. `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. -`plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to v2 engine. +`plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. `plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. From 902cae9fb528a0e91a850f3a8cd6b90924fbb24e Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:44:56 +0800 Subject: [PATCH 12/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 034dd743ab..2ec51b041d 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -84,7 +84,7 @@ Setting | Default | Description `plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. -`plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | The value is a factor to multiply the row count of the table scan to get the estimated row count. We recommend using the default value. +`plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | A factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. `plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. From 77f37010c6c695d297de117aacc5162f53254a80 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:45:12 +0800 Subject: [PATCH 13/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 2ec51b041d..d45d19c1e4 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -94,7 +94,7 @@ Setting | Default | Description `plugins.ppl.pattern.method` | `"simple_pattern"` | Sets the default patterns method. Another supported method is "brain". `plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. -`plugins.ppl.pattern.buffer.limit` | `100000` | Sets the default size of internal temporary buffer used in "brain" algorithm. +`plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. `plugins.ppl.query.timeout` | `5m` | Configures the maximum execution time for PPL queries. When a query exceeds this timeout, it will be interrupted and return a timeout error. From 1b7d76e3349d410498504ee882ea7915eaf2c868 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:45:25 +0800 Subject: [PATCH 14/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index d45d19c1e4..eac24ef6bc 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -95,7 +95,7 @@ Setting | Default | Description `plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. -`plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. +`plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered token output format. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. `plugins.ppl.query.timeout` | `5m` | Configures the maximum execution time for PPL queries. When a query exceeds this timeout, it will be interrupted and return a timeout error. From c85e511e1ee786d9988171f5dbae57c84b89c6b8 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:45:48 +0800 Subject: [PATCH 15/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 1 - 1 file changed, 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index eac24ef6bc..64810da205 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -96,7 +96,6 @@ Setting | Default | Description `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered token output format. -`plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to turn on numbered token output format. `plugins.ppl.query.timeout` | `5m` | Configures the maximum execution time for PPL queries. When a query exceeds this timeout, it will be interrupted and return a timeout error. ## Spark connector settings From 0ede38c8fda28f10a16c150b0e14167cbcf48627 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:46:23 +0800 Subject: [PATCH 16/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 64810da205..b044acf75b 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -96,7 +96,7 @@ Setting | Default | Description `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered token output format. -`plugins.ppl.query.timeout` | `5m` | Configures the maximum execution time for PPL queries. When a query exceeds this timeout, it will be interrupted and return a timeout error. +`plugins.ppl.query.timeout` | `5m` | Configures the maximum amount of time a PPL query can run. If a query exceeds this limit, execution stops and returns a timeout error. ## Spark connector settings From 29c0159f2093a66f08a00f691af2fed8cc7fb6f5 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:46:32 +0800 Subject: [PATCH 17/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index b044acf75b..1946160525 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -81,7 +81,7 @@ Setting | Default | Description `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. `plugins.query.field_type_tolerance` | `true` | If `false`, then an array is reduced to the first non-array value at any nesting level. For example, `[[1, 2], [3, 4]]` is reduced to `1`. If `true`, then the array is preserved. Default is `true`. `plugins.query.buckets` | `10000` | Sets the number of aggregation buckets returned in a single response. Default is the `plugins.query.size_limit` value. -`plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, join, and lookup operations. +`plugins.calcite.enabled` | `true` | Enables the Apache Calcite query engine, including advanced SQL and PPL capabilities such as subsearch, `join`, and `lookup` operations. `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | A factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. From fd884efacb7891a4e671d5d6e9e40c8ca5b0cf13 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:46:48 +0800 Subject: [PATCH 18/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 1946160525..7b02a92821 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -86,7 +86,7 @@ Setting | Default | Description `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | A factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. -`plugins.ppl.syntax.legacy.preferred` | `true` | Controls certain PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. +`plugins.ppl.syntax.legacy.preferred` | `true` | Controls PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. For more information, see the [legacy syntax documentation](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/settings.rst#plugins-ppl-syntax-legacy-preferred). `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. `plugins.ppl.rex.max_match.limit` | `10` | Maximum number of matches to extract in `rex` command. `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. From d4c33f22b993529731f99dcd9ef11359d10867da Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:47:00 +0800 Subject: [PATCH 19/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 7b02a92821..8e258a07ea 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -85,7 +85,7 @@ Setting | Default | Description `plugins.calcite.pushdown.enabled` | `true` | Change to `false` to disable the operator push-down optimization. We recommend using the default value. `plugins.calcite.fallback.allowed` | `false` | Change to `true` to allow fallback to the v2 engine. `plugins.calcite.pushdown.rowcount.estimation.factor` | `0.9` | A factor used to multiply the row count of a table scan to estimate the resulting row count. We recommend using the default value. -`plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, like `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. +`plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, such as `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. `plugins.ppl.syntax.legacy.preferred` | `true` | Controls PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. For more information, see the [legacy syntax documentation](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/settings.rst#plugins-ppl-syntax-legacy-preferred). `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. `plugins.ppl.rex.max_match.limit` | `10` | Maximum number of matches to extract in `rex` command. From 4b74c807753938be476ec152fcb4db0031dde4ae Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:47:27 +0800 Subject: [PATCH 20/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 8e258a07ea..a4cb6f1933 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -88,7 +88,7 @@ Setting | Default | Description `plugins.calcite.all_join_types.allowed` | `false` | Enables performance-sensitive join types, such as `RIGHT`, `FULL`, and `CROSS` joins. Change to `true` to allow these join operations. `plugins.ppl.syntax.legacy.preferred` | `true` | Controls PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. For more information, see the [legacy syntax documentation](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/settings.rst#plugins-ppl-syntax-legacy-preferred). `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. -`plugins.ppl.rex.max_match.limit` | `10` | Maximum number of matches to extract in `rex` command. +`plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. `plugins.ppl.pattern.method` | `"simple_pattern"` | Sets the default patterns method. Another supported method is "brain". From be22766feeb8e071810d13f18e8ba288c4475ffb Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:47:34 +0800 Subject: [PATCH 21/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index a4cb6f1933..c8b64bf5f2 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -91,7 +91,7 @@ Setting | Default | Description `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. -`plugins.ppl.pattern.method` | `"simple_pattern"` | Sets the default patterns method. Another supported method is "brain". +`plugins.ppl.pattern.method` | `simple_pattern` | Sets the method for the `patterns` command. Valid values are `simple_pattern` and `brain`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. From 1e13f6fd2c824a19118d45f9f70d149825f45a56 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:47:50 +0800 Subject: [PATCH 22/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index c8b64bf5f2..7fbd532fbb 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -92,7 +92,7 @@ Setting | Default | Description `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. `plugins.ppl.pattern.method` | `simple_pattern` | Sets the method for the `patterns` command. Valid values are `simple_pattern` and `brain`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). -`plugins.ppl.pattern.mode` | `"label"` | Sets the default patterns mode. Another supported mode is "aggregation". +`plugins.ppl.pattern.mode` | `label` | Sets the mode for the `patterns` command. Valid values are `label` and `aggregation`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered token output format. From 0c8a38a000d04ede1b8e11d9475ae6358e0c6b4d Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:48:12 +0800 Subject: [PATCH 23/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 7fbd532fbb..ea18e5a3ec 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -89,7 +89,7 @@ Setting | Default | Description `plugins.ppl.syntax.legacy.preferred` | `true` | Controls PPL syntax behaviors, including default argument values. When `false`, uses newer syntax standards. For more information, see the [legacy syntax documentation](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/settings.rst#plugins-ppl-syntax-legacy-preferred). `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. -`plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum of rows to return from subsearch. +`plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum number of rows to return from a subsearch. `plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. `plugins.ppl.pattern.method` | `simple_pattern` | Sets the method for the `patterns` command. Valid values are `simple_pattern` and `brain`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.mode` | `label` | Sets the mode for the `patterns` command. Valid values are `label` and `aggregation`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). From 5b965a342790432eaaec8e427ea8a4ae510fed3e Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 5 Dec 2025 11:48:22 +0800 Subject: [PATCH 24/25] Update _search-plugins/sql/settings.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Lantao Jin --- _search-plugins/sql/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index ea18e5a3ec..8a675108b4 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -90,7 +90,7 @@ Setting | Default | Description `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum number of rows to return from a subsearch. -`plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum of rows from subsearch to join against. +`plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum number of rows from to return from a subsearch used in a join operation. `plugins.ppl.pattern.method` | `simple_pattern` | Sets the method for the `patterns` command. Valid values are `simple_pattern` and `brain`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.mode` | `label` | Sets the mode for the `patterns` command. Valid values are `label` and `aggregation`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. From 168ea226351304f4f3363abc0c0c6d87ab6c5aba Mon Sep 17 00:00:00 2001 From: Nathan Bower Date: Fri, 5 Dec 2025 07:13:42 -0500 Subject: [PATCH 25/25] Apply suggestions from code review Signed-off-by: Nathan Bower --- _search-plugins/sql/settings.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index 8a675108b4..690509061a 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -75,7 +75,7 @@ Setting | Default | Description `plugins.sql.enabled` | `true` | Change to `false` to disable the `SQL` support in the plugin. `plugins.ppl.enabled` | `true` | Change to `false` to disable the `PPL` support in the plugin. `plugins.sql.slowlog` | `2` | Configures the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. -`plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource-intensive, we recommend a low value. +`plugins.sql.cursor.keep_alive` | `1m` | Configures the amount of time that the cursor context remains open. Because cursor contexts are resource intensive, we recommend a low value. `plugins.query.memory_limit` | `85%` | Configures the heap memory usage limit for the query engine's circuit breaker. `plugins.query.size_limit` | `10000` | Sets the maximum number of rows returned from a query execution. `plugins.query.datasources.enabled` | `true` | Change to `false` to disable support for data sources in the plugin. @@ -90,13 +90,13 @@ Setting | Default | Description `plugins.ppl.values.max.limit` | `0` | Sets the maximum number of unique values that the `VALUES` aggregation function can return. A value of `0` indicates no limit. `plugins.ppl.rex.max_match.limit` | `10` | Sets the maximum number of matches extracted by the `rex` command. `plugins.ppl.subsearch.maxout` | `10000` | Sets the maximum number of rows to return from a subsearch. -`plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum number of rows from to return from a subsearch used in a join operation. +`plugins.ppl.join.subsearch_maxout` | `50000` | Sets the maximum number of rows to return from a subsearch used in a join operation. `plugins.ppl.pattern.method` | `simple_pattern` | Sets the method for the `patterns` command. Valid values are `simple_pattern` and `brain`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.mode` | `label` | Sets the mode for the `patterns` command. Valid values are `label` and `aggregation`. For more information, see [`patterns` syntax](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/patterns.rst#syntax). `plugins.ppl.pattern.max.sample.count` | `10` | Sets the maximum number of sample logs returned per pattern in aggregation mode. `plugins.ppl.pattern.buffer.limit` | `100000` | Sets the size of the internal temporary buffer used by the `brain` algorithm. `plugins.ppl.pattern.show.numbered.token` | `false` | Change to `true` to enable the numbered token output format. -`plugins.ppl.query.timeout` | `5m` | Configures the maximum amount of time a PPL query can run. If a query exceeds this limit, execution stops and returns a timeout error. +`plugins.ppl.query.timeout` | `5m` | Configures the maximum amount of time that a PPL query can run. If a query exceeds this limit, execution stops and returns a timeout error. ## Spark connector settings