Skip to content

Commit 9f48101

Browse files
ZhouXing19ZhouXing19
committed
sql/opt: add cluster setting sql.stats.canary_fraction and session var canary_stats_mode
This commit introduces mechanisms to control the use of canary statistics in query planning. For any non-internal query, we generate a random number within [0, 1] and compare with sql.stats.canary_fraction to decide if the query planning should use canary stats or not. Since this "dice roll" happens for every non-internal query, the memo would otherwise flip frequently, negating the benefits of the query plan cache and causing performance regressions. To mitigate this, queries selected for the canary path bypass the query plan cache entirely: they neither look up existing cached memos nor invalidate them. Instead, we create a one-time memo used only for that single query execution. This approach assumes sql.stats.canary_fraction will be set to a small value, ensuring that canary queries remain a small fraction of total queries and minimizing the performance impact of recomputation. Release note (sql change): This release introduces two new settings to control the use of canary statistics in query planning: 1. Cluster setting `sql.stats.canary_fraction` (float, range [0, 1], default: 0): Controls what fraction of queries use "canary statistics" (newly collected stats within their canary window) versus "stable statistics" (previously proven stats). For example, a value of 0.2 means 20% of queries will use canary stats while 80% use stable stats. The selection is atomic per query: if a query is chosen for canary evaluation, it uses canary statistics for ALL tables it references (where available). A query never uses a mix of canary and stable statistics. 2. Session variable `canary_stats_mode` (enum: {auto, off, on}, default: auto): - `on`: All queries in the session use canary stats for planning - `off`: All queries in the session use stable stats for planning - `auto`: The system decides based on `sql.stats.canary_fraction` for each query execution
1 parent a8567ec commit 9f48101

File tree

14 files changed

+442
-4
lines changed

14 files changed

+442
-4
lines changed

‎docs/generated/settings/settings-for-tenants.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,6 +361,7 @@ sql.stats.automatic_full_collection.enabled boolean true automatic full statisti
361361
sql.stats.automatic_partial_collection.enabled boolean true automatic partial statistics collection mode application
362362
sql.stats.automatic_partial_collection.fraction_stale_rows float 0.05 target fraction of stale rows per table that will trigger a partial statistics refresh application
363363
sql.stats.automatic_partial_collection.min_stale_rows integer 100 target minimum number of stale rows per table that will trigger a partial statistics refresh application
364+
sql.stats.canary_fraction float 0 probability that table statistics will use canary mode instead of stable mode for query planning [0.0-1.0] application
364365
sql.stats.cleanup.recurrence string @hourly cron-tab recurrence for SQL Stats cleanup job application
365366
sql.stats.detailed_latency_metrics.enabled boolean false label latency metrics with the statement fingerprint. Workloads with tens of thousands of distinct query fingerprints should leave this setting false. (experimental, affects performance for workloads with high fingerprint cardinality) application
366367
sql.stats.error_on_concurrent_create_stats.enabled boolean false set to true to error on concurrent CREATE STATISTICS jobs, instead of skipping them application

‎docs/generated/settings/settings.html‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -316,6 +316,7 @@
316316
<tr><td><div id="setting-sql-stats-automatic-partial-collection-enabled" class="anchored"><code>sql.stats.automatic_partial_collection.enabled</code></div></td><td>boolean</td><td><code>true</code></td><td>automatic partial statistics collection mode</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
317317
<tr><td><div id="setting-sql-stats-automatic-partial-collection-fraction-stale-rows" class="anchored"><code>sql.stats.automatic_partial_collection.fraction_stale_rows</code></div></td><td>float</td><td><code>0.05</code></td><td>target fraction of stale rows per table that will trigger a partial statistics refresh</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
318318
<tr><td><div id="setting-sql-stats-automatic-partial-collection-min-stale-rows" class="anchored"><code>sql.stats.automatic_partial_collection.min_stale_rows</code></div></td><td>integer</td><td><code>100</code></td><td>target minimum number of stale rows per table that will trigger a partial statistics refresh</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
319+
<tr><td><div id="setting-sql-stats-canary-fraction" class="anchored"><code>sql.stats.canary_fraction</code></div></td><td>float</td><td><code>0</code></td><td>probability that table statistics will use canary mode instead of stable mode for query planning [0.0-1.0]</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
319320
<tr><td><div id="setting-sql-stats-cleanup-recurrence" class="anchored"><code>sql.stats.cleanup.recurrence</code></div></td><td>string</td><td><code>@hourly</code></td><td>cron-tab recurrence for SQL Stats cleanup job</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
320321
<tr><td><div id="setting-sql-stats-detailed-latency-metrics-enabled" class="anchored"><code>sql.stats.detailed_latency_metrics.enabled</code></div></td><td>boolean</td><td><code>false</code></td><td>label latency metrics with the statement fingerprint. Workloads with tens of thousands of distinct query fingerprints should leave this setting false. (experimental, affects performance for workloads with high fingerprint cardinality)</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
321322
<tr><td><div id="setting-sql-stats-error-on-concurrent-create-stats-enabled" class="anchored"><code>sql.stats.error_on_concurrent_create_stats.enabled</code></div></td><td>boolean</td><td><code>false</code></td><td>set to true to error on concurrent CREATE STATISTICS jobs, instead of skipping them</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>

‎pkg/sql/conn_executor.go‎

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,57 @@ var detailedLatencyMetrics = settings.RegisterBoolSetting(
156156
settings.WithPublic,
157157
)
158158

159+
// canaryFraction controls the probabilistic sampling rate for queries
160+
// participating in the canary statistics rollout feature.
161+
//
162+
// This cluster-level setting determines what fraction of queries will use
163+
// "canary statistics" (newly collected stats within their canary window)
164+
// versus "stable statistics" (previously proven stats). For example, a value
165+
// of 0.2 means 20% of queries will test canary stats while 80% use stable stats.
166+
//
167+
// The selection is atomic per query: if a query is chosen for canary evaluation,
168+
// it will use canary statistics for ALL tables it references (where available).
169+
// A query never uses a mix of canary and stable statistics.
170+
var canaryFraction = settings.RegisterFloatSetting(
171+
settings.ApplicationLevel,
172+
"sql.stats.canary_fraction",
173+
"probability that table statistics will use canary mode instead of stable mode for query planning [0.0-1.0]",
174+
0,
175+
settings.Fraction,
176+
settings.WithPublic,
177+
)
178+
179+
// canaryRollDice performs the probabilistic check to determine if a query
180+
// should use the "canary path" for statistics.
181+
// This selection is atomic per query, meaning that if a query is chosen to use
182+
// canary stats, all the tables involved in this query will use canary stats for
183+
// planning, if applicable.
184+
// This canary stats decision is made only for non-internal queries.
185+
func canaryRollDice(evalCtx *eval.Context, rng *rand.Rand) bool {
186+
switch m := evalCtx.SessionData().CanaryStatsMode; m {
187+
case sessiondatapb.CanaryStatsModeAuto:
188+
threshold := canaryFraction.Get(&evalCtx.Settings.SV)
189+
// If the fraction is 0, never use canary stats.
190+
if threshold == 0 {
191+
return false
192+
}
193+
// If the fraction is 1, always use canary stats.
194+
if threshold == 1 {
195+
return true
196+
}
197+
198+
actual := rng.Float64()
199+
return actual < threshold
200+
case sessiondatapb.CanaryStatsModeOff:
201+
return false
202+
case sessiondatapb.CanaryStatsModeOn:
203+
return true
204+
}
205+
// This should not happen but just in case of an unknown mode, we
206+
// default to not using canary stats.
207+
return false
208+
}
209+
159210
// The metric label name we'll use to facet latency metrics by statement fingerprint.
160211
var detailedLatencyMetricLabel = "fingerprint"
161212

@@ -1804,7 +1855,8 @@ type connExecutor struct {
18041855
// rng contains random number generators for this session.
18051856
rng struct {
18061857
// internal is used for internal operations like determining the query
1807-
// cancel key and whether sampling execution stats should be performed.
1858+
// cancel key, whether sampling execution stats should be performed, and
1859+
// whether the query should use canary stats versus stable stats.
18081860
internal *rand.Rand
18091861
// external is used to power random() builtin. It is important to store
18101862
// this field by value so that the same RNG is reused throughout the
@@ -3934,6 +3986,7 @@ func (ex *connExecutor) resetEvalCtx(evalCtx *extendedEvalContext, txn *kv.Txn,
39343986
evalCtx.SkipNormalize = false
39353987
evalCtx.SchemaChangerState = ex.extraTxnState.schemaChangerState
39363988
evalCtx.DescIDGenerator = ex.getDescIDGenerator()
3989+
evalCtx.UseCanaryStats = false
39373990

39383991
// See resetPlanner for more context on setting the maximum timestamp for
39393992
// AOST read retries.

‎pkg/sql/conn_executor_exec.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3226,6 +3226,12 @@ func (ex *connExecutor) makeExecPlan(
32263226
return ctx, err
32273227
}
32283228

3229+
// For each non-internal query, we roll the dice to decide to use
3230+
// canary stats or stable stats for planning.
3231+
if !planner.SessionData().Internal {
3232+
planner.EvalContext().UseCanaryStats = canaryRollDice(planner.EvalContext(), ex.rng.internal)
3233+
}
3234+
32293235
if err := planner.makeOptimizerPlan(ctx); err != nil {
32303236
log.VEventf(ctx, 1, "optimizer plan failed: %v", err)
32313237
return ctx, err

‎pkg/sql/logictest/testdata/logic_test/canary_stats‎

Lines changed: 267 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ SET create_table_with_schema_locked=false
99

1010
statement ok
1111
CREATE TABLE canary_table (x int primary key, y int, FAMILY (x, y)) WITH (sql_stats_canary_window = '20s');
12+
INSERT INTO canary_table VALUES (1, 1);
1213

1314
query TT
1415
SHOW CREATE TABLE canary_table
@@ -45,3 +46,269 @@ CREATE TABLE public.canary_table (
4546
CONSTRAINT canary_table_pkey PRIMARY KEY (x ASC),
4647
FAMILY fam_0_x_y (x, y)
4748
) WITH (sql_stats_canary_window = '20s')
49+
50+
subtest canary_stats_with_query_cache
51+
52+
# Test with stable stats, which should have the normal usage of query cache.
53+
statement ok
54+
SET CLUSTER SETTING sql.stats.canary_fraction='0';
55+
56+
statement ok
57+
SET TRACING = "on", cluster;
58+
59+
statement ok
60+
SELECT * FROM canary_table;
61+
62+
statement ok
63+
SET TRACING = "off";
64+
65+
# The first execution should miss the query cache.
66+
query T
67+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%';
68+
----
69+
query cache miss
70+
query cache add
71+
72+
statement ok
73+
SET TRACING = "on", cluster;
74+
75+
statement ok
76+
SELECT * FROM canary_table;
77+
78+
statement ok
79+
SET TRACING = "off";
80+
81+
# The second execution should hit the query cache.
82+
query T
83+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%';
84+
----
85+
query cache hit
86+
87+
# Now we test the case where always use canary stats, which doesn't
88+
# interact with query cache at all.
89+
statement ok
90+
SET CLUSTER SETTING sql.stats.canary_fraction='1.0';
91+
92+
statement ok
93+
SET TRACING = "on", cluster;
94+
95+
statement ok
96+
SELECT * FROM canary_table;
97+
98+
statement ok
99+
SET TRACING = "off";
100+
101+
query T
102+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%';
103+
----
104+
not using query cache
105+
106+
statement ok
107+
SET TRACING = "on", cluster;
108+
109+
statement ok
110+
SELECT * FROM canary_table;
111+
112+
statement ok
113+
SET TRACING = "off";
114+
115+
query T
116+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%';
117+
----
118+
not using query cache
119+
120+
# Switching back to using stable stats, and the same query should still be
121+
# able to hit the query cache.
122+
statement ok
123+
SET CLUSTER SETTING sql.stats.canary_fraction='0.0';
124+
125+
statement ok
126+
SET TRACING = "on", cluster;
127+
128+
statement ok
129+
SELECT * FROM canary_table;
130+
131+
statement ok
132+
SET TRACING = "off";
133+
134+
# We should see the original cache is still there.
135+
query T
136+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%';
137+
----
138+
query cache hit
139+
140+
subtest end
141+
142+
subtest prepared_statements
143+
144+
statement ok
145+
SET CLUSTER SETTING sql.stats.canary_fraction='1.0';
146+
147+
statement ok
148+
PREPARE p1 AS SELECT * FROM canary_table WHERE x = $1;
149+
150+
query II
151+
EXECUTE p1(1);
152+
----
153+
1 1
154+
155+
# Test with prepared statement without placeholder.
156+
statement ok
157+
PREPARE p2 AS SELECT * FROM canary_table WHERE x = 1;
158+
159+
query II
160+
EXECUTE p2;
161+
----
162+
1 1
163+
164+
statement ok
165+
SET CLUSTER SETTING sql.stats.canary_fraction='0';
166+
167+
subtest end
168+
169+
subtest prepared_statements_with_query_cache
170+
171+
statement ok
172+
DEALLOCATE ALL;
173+
174+
statement ok
175+
SELECT crdb_internal.clear_query_plan_cache();
176+
177+
statement ok
178+
SET TRACING = "on", cluster;
179+
180+
# We are using stable stats for every query, so the query cache should be used.
181+
statement ok
182+
PREPARE p1 AS SELECT * from canary_table;
183+
PREPARE p2 AS SELECT * from canary_table;
184+
PREPARE p3 AS SELECT * from canary_table WHERE x = $1;
185+
PREPARE p4 AS SELECT * from canary_table WHERE x = $1;
186+
187+
statement ok
188+
SET TRACING = "off";
189+
190+
query T
191+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%' OR message LIKE '%cached memo%';
192+
----
193+
query cache miss (prepare)
194+
query cache hit (prepare)
195+
query cache miss (prepare)
196+
query cache hit (prepare)
197+
198+
statement ok
199+
SET TRACING = "on", cluster;
200+
201+
statement ok
202+
EXECUTE p1;
203+
EXECUTE p2;
204+
EXECUTE p3(1);
205+
EXECUTE p3(1);
206+
EXECUTE p4(1);
207+
EXECUTE p4(1);
208+
209+
statement ok
210+
SET TRACING = "off";
211+
212+
query T
213+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%' OR message LIKE '%cached memo%';
214+
----
215+
reusing cached memo
216+
reusing cached memo
217+
reusing cached memo
218+
reusing cached memo
219+
reusing cached memo
220+
reusing cached memo
221+
222+
# We are using canary stats for all queries now, but we don't roll the dice
223+
# for prepared statement. I.e. the prepare stmt would still be cached,
224+
# and memo would still be built with stable stats and cached within the prepared
225+
# statement.
226+
statement ok
227+
SET CLUSTER SETTING sql.stats.canary_fraction='1.0';
228+
229+
statement ok
230+
SET TRACING = "on", cluster;
231+
232+
# p5, p6 are based on the same queries as p1, p3 respectively, so they
233+
# should hit the query cache. p7 is a new query so it should see a query
234+
# cache miss, but itself would be cached. p8 is based on the same query as
235+
# p7, so it should hit the query cache.
236+
statement ok
237+
PREPARE p5 AS SELECT * from canary_table;
238+
PREPARE p6 AS SELECT * from canary_table WHERE x = $1;
239+
PREPARE p7 AS SELECT * from canary_table WHERE y = $1;
240+
PREPARE p8 AS SELECT * from canary_table WHERE y = $1;
241+
242+
statement ok
243+
SET TRACING = "off";
244+
245+
query T
246+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%' OR message LIKE '%cached memo%';
247+
----
248+
query cache hit (prepare)
249+
query cache hit (prepare)
250+
query cache miss (prepare)
251+
query cache hit (prepare)
252+
253+
statement ok
254+
SET TRACING = "on", cluster;
255+
256+
# For execution, we roll the dice for every one. Since we have canary_fraction=1.0,
257+
# all execution will not use the query cache nor the cached memo within
258+
# the prepared statement. Thus we should see "not using query cache" for all executions.
259+
statement ok
260+
EXECUTE p5;
261+
EXECUTE p5;
262+
EXECUTE p6(1);
263+
EXECUTE p6(1);
264+
EXECUTE p3(1);
265+
EXECUTE p3(1);
266+
EXECUTE p7(1);
267+
EXECUTE p7(1);
268+
EXECUTE p8(1);
269+
EXECUTE p8(1);
270+
271+
statement ok
272+
SET TRACING = "off";
273+
274+
query T
275+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%' OR message LIKE '%cached memo%';
276+
----
277+
not using query cache
278+
not using query cache
279+
not using query cache
280+
not using query cache
281+
not using query cache
282+
not using query cache
283+
not using query cache
284+
not using query cache
285+
not using query cache
286+
not using query cache
287+
288+
# We now switch back to the "everyone use stable stats" mode. At this moment
289+
# executing a prepared stmt that is created while canary_fration == 1.0 should
290+
# still be able to use the cached memo.
291+
statement ok
292+
SET CLUSTER SETTING sql.stats.canary_fraction='0';
293+
294+
statement ok
295+
SET TRACING = "on", cluster;
296+
297+
statement ok
298+
EXECUTE p7(1);
299+
EXECUTE p7(1);
300+
EXECUTE p8(1);
301+
EXECUTE p8(1);
302+
303+
statement ok
304+
SET TRACING = "off";
305+
306+
query T
307+
SELECT message FROM [SHOW TRACE FOR SESSION] WHERE message LIKE '%query cache%' OR message LIKE '%cached memo%';
308+
----
309+
reusing cached memo
310+
reusing cached memo
311+
reusing cached memo
312+
reusing cached memo
313+
314+
subtest end

‎pkg/sql/logictest/testdata/logic_test/information_schema‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4088,6 +4088,7 @@ backslash_quote safe_encoding
40884088
buffered_writes_use_locking_on_non_unique_indexes off
40894089
bypass_pcr_reader_catalog_aost off
40904090
bytea_output hex
4091+
canary_stats_mode auto
40914092
check_function_bodies on
40924093
client_encoding UTF8
40934094
client_min_messages notice

0 commit comments

Comments
 (0)