Skip to content

Commit bb329fc

Browse files
ddhodgeddhodge
authored
[DOC-985] Move auto analyze to additional capabilities (#28691)
* Move auto analyze to additional capabilities
1 parent 09b80ee commit bb329fc

File tree

18 files changed

+88
-51
lines changed

18 files changed

+88
-51
lines changed

‎docs/content/preview/additional-features/_index.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,4 +38,10 @@ type: indexpage
3838
href="pg-extensions/"
3939
icon="fa-thin fa-puzzle">}}
4040

41+
{{<index/item
42+
title="Auto Analyze service"
43+
body="Automate execution of ANALYZE commands."
44+
href="auto-analyze/"
45+
icon="fa-thin fa-microscope">}}
46+
4147
{{</index/block>}}

‎docs/content/preview/explore/query-1-performance/auto-analyze.md‎ renamed to ‎docs/content/preview/additional-features/auto-analyze.md‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,26 +9,30 @@ tags:
99
menu:
1010
preview:
1111
identifier: auto-analyze
12-
parent: query-tuning
13-
weight: 700
12+
parent: additional-features
13+
weight: 100
14+
aliases:
15+
- /preview/explore/query-1-performance/auto-analyze/
1416
type: docs
1517
---
1618

17-
To create optimal plans for queries, the query planner needs accurate and up-to-date statistics related to tables and their columns. These statistics are also used by the YugabyteDB [cost-based optimizer](../../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) (CBO) to create optimal execution plans for queries. To generate the statistics, you run the [ANALYZE](../../../api/ysql/the-sql-language/statements/cmd_analyze/) command. ANALYZE collects statistics about the contents of tables in the database, and stores the results in the `pg_statistic` system catalog.
19+
To create optimal plans for queries, the query planner needs accurate and up-to-date statistics related to tables and their columns. These statistics are also used by the YugabyteDB [cost-based optimizer](../../reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model) (CBO) to create optimal execution plans for queries. To generate the statistics, you run the [ANALYZE](../../api/ysql/the-sql-language/statements/cmd_analyze/) command. ANALYZE collects statistics about the contents of tables in the database, and stores the results in the `pg_statistic` system catalog.
1820

1921
Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM), the YugabyteDB Auto Analyze service automates the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold for the table. This ensures table statistics are always up-to-date.
2022

2123
## Enable Auto Analyze
2224

2325
Before you can use the feature, you must enable it by setting `ysql_enable_auto_analyze_service` to true on all YB-Masters, and both `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true on all YB-TServers.
2426

25-
For example, to create a single-node [yugabyted](../../../reference/configuration/yugabyted/) cluster with Auto Analyze enabled, use the following command:
27+
For example, to create a single-node [yugabyted](../../reference/configuration/yugabyted/) cluster with Auto Analyze enabled, use the following command:
2628

2729
```sh
28-
./bin/yugabyted start --master_flags "ysql_enable_auto_analyze_service=true" --tserver_flags "ysql_enable_auto_analyze_service=true,ysql_enable_table_mutation_counter=true"
30+
./bin/yugabyted start \
31+
--master_flags "ysql_enable_auto_analyze_service=true" \
32+
--tserver_flags "ysql_enable_auto_analyze_service=true,ysql_enable_table_mutation_counter=true"
2933
```
3034

31-
To enable Auto Analyze on an existing cluster, a rolling restart is required to set `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true.
35+
Enabling Auto Analyze on an existing cluster requires a rolling restart to set `ysql_enable_auto_analyze_service` and `ysql_enable_table_mutation_counter` to true.
3236

3337
## Configure Auto Analyze
3438

@@ -53,7 +57,7 @@ On the other hand, `ysql_auto_analyze_scale_factor` is especially important for
5357

5458
In addition, `ysql_auto_analyze_batch_size` controls the maximum number of tables the Auto Analyze service tries to analyze in a single ANALYZE statement. The default is 10. Setting this flag to a larger value can potentially reduce the number of YSQL catalog cache refreshes if Auto Analyze decides to ANALYZE many tables in the same database at the same time.
5559

56-
For more information on flags used to configure the Auto Analyze service, refer to [Auto Analyze service flags](../../../reference/configuration/yb-tserver#auto-analyze-service-flags).
60+
For more information on flags used to configure the Auto Analyze service, refer to [Auto Analyze service flags](../../reference/configuration/yb-tserver/#auto-analyze-service-flags).
5761

5862
## Example
5963

@@ -73,7 +77,11 @@ SELECT reltuples FROM pg_class WHERE relname = 'test';
7377

7478
```sql
7579
INSERT INTO test SELECT i, i FROM generate_series(1, 100) i;
76-
-- Wait for few seconds
80+
```
81+
82+
Wait for few seconds.
83+
84+
```sql
7785
SELECT reltuples FROM pg_class WHERE relname = 'test';
7886
```
7987

‎docs/content/preview/api/ysql/the-sql-language/statements/cmd_analyze.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ The statistics are also used by the YugabyteDB [cost based optimizer](../../../.
2121
{{< warning title="Run ANALYZE regularly" >}}
2222
If you have enabled CBO, you must run ANALYZE on user tables after data load for the CBO to create optimal execution plans.
2323

24-
You can automate running ANALYZE using the [Auto Analyze service](../../../../../explore/query-1-performance/auto-analyze/).
24+
You can automate running ANALYZE using the [Auto Analyze service](../../../../../additional-features/auto-analyze/).
2525
{{< /warning >}}
2626

2727
The YugabyteDB implementation is based on the framework provided by PostgreSQL, which requires the storage layer to provide a random sample of rows of a predefined size. The size is calculated based on a number of factors, such as the included columns' data types.

‎docs/content/preview/architecture/query-layer/planner-optimizer.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ To optimize the search for the best plan, the CBO uses a dynamic programming-bas
3939

4040
The optimizer relies on accurate statistics about the tables, including the number of rows, the distribution of data in columns, and the cardinality of results from operations. These statistics are essential for estimating the selectivity of filters and costs of various query plans accurately. These statistics are gathered by the [ANALYZE](../../../api/ysql/the-sql-language/statements/cmd_analyze/) command and are provided in a display-friendly format by the [pg_stats](../../../architecture/system-catalog/#data-statistics) view.
4141

42-
Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM), the YugabyteDB [Auto Analyze](../../../explore/query-1-performance/auto-analyze/) service automates the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold for the table. This ensures table statistics are always up-to-date.
42+
Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM), the YugabyteDB [Auto Analyze](../../../additional-features/auto-analyze/) service automates the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold for the table. This ensures table statistics are always up-to-date.
4343

4444
Even with the Auto Analyze service, for the CBO to create optimal execution plans, you should still run ANALYZE manually on user tables after data load, as well as in other circumstances. Refer to [Best practices](#best-practices).
4545

‎docs/content/preview/best-practices-operations/ysql-yb-enable-cbo.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,8 @@ If your tables are analyzed, do the following:
9696

9797
## ANALYZE and Auto Analyze service
9898

99-
Use the [YugabyteDB Auto Analyze service](../../explore/query-1-performance/auto-analyze/) {{<tags/feature/ea idea="590">}} to automate the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold. This ensures table statistics are always up-to-date.
99+
Use the [YugabyteDB Auto Analyze service](../../additional-features/auto-analyze/) {{<tags/feature/ea idea="590">}} to automate the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold. This ensures table statistics are always up-to-date.
100100

101101
Even with the Auto Analyze service, for the CBO to create optimal execution plans, you should still run ANALYZE manually on user tables after data load, as well as in other circumstances.
102102

103-
For more information, refer to [Auto Analyze](../../explore/query-1-performance/auto-analyze/).
103+
For more information, refer to [Auto Analyze](../../additional-features/auto-analyze/).

‎docs/content/preview/explore/query-1-performance/_index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,8 @@ Learn more about [YB-TServer logs](/preview/explore/observability/logging/).
7171

7272
To create optimal plans for queries, the query planner needs accurate and up-to-date statistics related to tables and their columns. ANALYZE collects statistics about the contents of tables in the database, and stores the results in the `pg_statistic` system catalog. Similar to [PostgreSQL autovacuum](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM), the YugabyteDB Auto Analyze service automates the execution of ANALYZE commands for any table where rows have changed more than a configurable threshold for the table. This ensures table statistics are always up-to-date.
7373

74-
{{<lead link="./auto-analyze/">}}
75-
To learn more, see [Auto Analyze service](./auto-analyze/).
74+
{{<lead link="../../additional-features/auto-analyze/">}}
75+
To learn more, see [Auto Analyze service](../../additional-features/auto-analyze/).
7676
{{</lead>}}
7777

7878
## Export query diagnostics

‎docs/content/preview/explore/query-1-performance/pg-stats.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,11 @@ menu:
1111
type: docs
1212
---
1313

14-
[pg_stats](../../../architecture/system-catalog#data-statistics) is a system view that provides statistical information about columns in tables. Use it to view data distributions of column values, which can help in query optimization and tuning. To generate the statistics, run the [ANALYZE](../../../api/ysql/the-sql-language/statements/cmd_analyze/) command.
14+
[pg_stats](../../../architecture/system-catalog#data-statistics) is a system view that provides statistical information about columns in tables. Use it to view data distributions of column values, which can help in query optimization and tuning.
15+
16+
To generate the statistics, run the [ANALYZE](../../../api/ysql/the-sql-language/statements/cmd_analyze/) command.
17+
18+
To automatically update the statistics, configure the [Auto Analyze service](../../../additional-features/auto-analyze/).
1519

1620
## Key statistics
1721

‎docs/content/preview/reference/configuration/yb-master.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1042,7 +1042,7 @@ Default: `true`
10421042

10431043
## Auto Analyze service flags
10441044

1045-
Auto analyze is {{<tags/feature/ea idea="590">}}.
1045+
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
10461046

10471047
See also [Auto Analyze Service TServer flags](../yb-tserver/#auto-analyze-service-flags).
10481048

‎docs/content/preview/reference/configuration/yb-tserver.md‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2106,20 +2106,22 @@ For information on using this parameter to configure CBO, refer to [Enable cost-
21062106
21072107
### Auto Analyze service flags
21082108
2109-
Auto analyze is {{<tags/feature/ea idea="590">}}.
2109+
{{<tags/feature/ea idea="590">}}To learn about the Auto Analyze service, see [Auto Analyze service](../../../additional-features/auto-analyze).
21102110
21112111
{{< note title="Note" >}}
21122112
21132113
To fully enable the Auto Analyze service, you need to enable `ysql_enable_auto_analyze_service` on all YB-Masters and YB-TServers, and `ysql_enable_table_mutation_counter` on all YB-TServers.
21142114
21152115
{{< /note >}}
21162116
2117-
See also [Auto Analyze Service Master flags](../yb-master#auto-analyze-service-flags).
2117+
See also [Auto Analyze Service Master flags](../yb-master/#auto-analyze-service-flags).
21182118
21192119
##### ysql_enable_auto_analyze_service
21202120
21212121
{{% tags/wrap %}}
21222122
{{<tags/feature/ea idea="590">}}
2123+
{{<tags/feature/t-server>}}
2124+
{{<tags/feature/restart-needed>}}
21232125
Default: `false`
21242126
{{% /tags/wrap %}}
21252127
@@ -2129,7 +2131,6 @@ Enable the Auto Analyze service, which automatically runs ANALYZE to update tabl
21292131
21302132
{{% tags/wrap %}}
21312133
2132-
21332134
Default: `false`
21342135
{{% /tags/wrap %}}
21352136
@@ -2139,7 +2140,7 @@ Enable per table mutation (INSERT, UPDATE, DELETE) counting. The Auto Analyze se
21392140
21402141
{{% tags/wrap %}}
21412142
2142-
2143+
{{<tags/feature/restart-needed>}}
21432144
Default: `50`
21442145
{{% /tags/wrap %}}
21452146
@@ -2149,7 +2150,7 @@ The minimum number of mutations needed to run ANALYZE on a table.
21492150
21502151
{{% tags/wrap %}}
21512152
2152-
2153+
{{<tags/feature/restart-needed>}}
21532154
Default: `0.1`
21542155
{{% /tags/wrap %}}
21552156
@@ -2161,7 +2162,7 @@ ANALYZE runs when the mutation count exceeds `ysql_auto_analyze_scale_factor * <
21612162
21622163
{{% tags/wrap %}}
21632164
2164-
2165+
{{<tags/feature/restart-needed>}}
21652166
Default: `10`
21662167
{{% /tags/wrap %}}
21672168
@@ -2171,7 +2172,7 @@ The maximum number of tables the Auto Analyze service tries to analyze in a sing
21712172
21722173
{{% tags/wrap %}}
21732174
2174-
2175+
{{<tags/feature/restart-needed>}}
21752176
Default: `10000`
21762177
{{% /tags/wrap %}}
21772178
@@ -2181,7 +2182,7 @@ Interval at which the reported node level table mutation counts are persisted to
21812182
21822183
{{% tags/wrap %}}
21832184
2184-
2185+
{{<tags/feature/restart-needed>}}
21852186
Default: `10000`
21862187
{{% /tags/wrap %}}
21872188
@@ -2191,7 +2192,7 @@ Timeout for the RPCs used to persist mutation counts in the auto-analyze mutatio
21912192
21922193
{{% tags/wrap %}}
21932194
2194-
2195+
{{<tags/feature/restart-needed>}}
21952196
Default: `5000`
21962197
{{% /tags/wrap %}}
21972198
@@ -2201,7 +2202,7 @@ Interval, in milliseconds, at which the node-level table mutation counts are sen
22012202
22022203
{{% tags/wrap %}}
22032204
2204-
2205+
{{<tags/feature/restart-needed>}}
22052206
Default: `5000`
22062207
{{% /tags/wrap %}}
22072208

‎docs/content/stable/additional-features/_index.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,4 +38,10 @@ type: indexpage
3838
href="pg-extensions/"
3939
icon="fa-thin fa-puzzle">}}
4040

41+
{{<index/item
42+
title="Auto Analyze service"
43+
body="Automate execution of ANALYZE commands."
44+
href="auto-analyze/"
45+
icon="fa-thin fa-microscope">}}
46+
4147
{{</index/block>}}

0 commit comments

Comments
 (0)