Merge #142927 #143523 #143578 #143825

142927: builtins,workloadindexrec: Add fingerprint id to workload_index_recs … r=kyle-a-wong a=kyle-a-wong

…builtin

Adds another column to the workload_index_recs builtin that contains an array of fingerprint ids that the recommended indexes were originally recommended for.

Resolves: #141947
Epic: CRDB-42980
Release note (sql change): Changes the return type of the workload_index_recs builtin to be two columns. The first column, "index_rec", remains a string type that contains the index recommndation. The second column, "fingerprint_ids", is new and has the "byte[]" type.

143523: sql: validate undesired zone configs for secondary tenants r=annrpom a=annrpom

### sql: add setting for constraint validation for secondary tenants
This patch adds a cluster setting, ``sql.zone_configs.max_replicas_per_region,
that helps prevent virtual clusters from setting undesired zone configs. This
setting defines the max number of replicas that can be configured for a
single region (0 for unlimited).

Epic: none
Informs: #142856

Release note: None

---

### sql: add setting to validate privs for secondary tenants
This patch adds a cluster setting,
`sql.zone_configs.default_range_modifiable_by_non_root.enabled`, to
help prevent virtual clusters from setting undesired zone configs.
This settings determines if non-root users on these clusters should
be restricted from modifying the `default` range. Other named ranges
are not allowed to be modified as a secondary tenant.

Epic: none
Fixes: #142856

Release note: None

143578: kvserver: don't generate large writes during lock durability upgrade r=arulajmani a=stevendanna

Since this is still a best-effort feature for the time being, we want to protect against writing too many locks during transfer or merge requests.

Epic: none
Release note: None

143825: roachtest/pg_regress: accept recent diff r=yuzefovich a=yuzefovich

**roachtest/pg_regress: adjust runner to remove table IDs**

I've seen a particular unimplemented error result in the diffs churn
because it includes table IDs which can change between run, for some
reason. Let's just remove the table ID from the error.


**roachtest/pg_regress: accept recent diff**

Fixes: #143940.

Release note: None

Co-authored-by: Kyle Wong <[email protected]>
Co-authored-by: Annie Pompa <[email protected]>
Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>

Author: 5 people

docs/generated/sql/functions.md

@@ -1554,9 +1554,9 @@ the locality flag on node startup. Returns an error if no region is set.</p>
 </span></td><td>Immutable</td></tr>
 <tr><td><a name="unnest"></a><code>unnest(input: anyelement[]) &rarr; anyelement</code></td><td><span class="funcdesc"><p>Returns the input array as a set of rows</p>
 </span></td><td>Immutable</td></tr>
-<tr><td><a name="workload_index_recs"></a><code>workload_index_recs() &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
+<tr><td><a name="workload_index_recs"></a><code>workload_index_recs() &rarr; tuple{string AS index_rec, bytes[] AS fingerprint_ids}</code></td><td><span class="funcdesc"><p>Returns index recommendations and the fingerprint ids that the indexes will impact</p>
 </span></td><td>Immutable</td></tr>
-<tr><td><a name="workload_index_recs"></a><code>workload_index_recs(timestamptz: <a href="timestamp.html">timestamptz</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
+<tr><td><a name="workload_index_recs"></a><code>workload_index_recs(timestamptz: <a href="timestamp.html">timestamptz</a>) &rarr; tuple{string AS index_rec, bytes[] AS fingerprint_ids}</code></td><td><span class="funcdesc"><p>Returns index recommendations and the fingerprint ids that the indexes will impact</p>
 </span></td><td>Immutable</td></tr></tbody>
 </table>
 

pkg/ccl/logictestccl/testdata/logic_test/multi_region_secondary_tenants_abstractions_allowed

@@ -0,0 +1,85 @@
+# LogicTest: multiregion-9node-3region-3azs-tenant
+# tenant-cluster-setting-override-opt: sql.virtual_cluster.feature_access.multiregion.enabled=true
+
+statement ok
+CREATE DATABASE db PRIMARY REGION "us-east-1"
+
+statement ok
+ALTER DATABASE db ADD REGION "ca-central-1"
+
+statement ok
+SET override_multi_region_zone_config = true
+
+statement ok
+ALTER DATABASE db CONFIGURE ZONE USING constraints = '{+region=us-east-1: 4}', voter_constraints = '{+region=us-east-1: 3}'
+
+subtest constrained_replicas
+
+user root
+
+statement ok
+SET CLUSTER SETTING sql.zone_configs.max_replicas_per_region = 10
+
+user host-cluster-root
+
+# Ensure that the override from the system tenant takes precedence over anything
+# set from within the application tenant.
+statement ok
+ALTER TENANT ALL SET CLUSTER SETTING sql.zone_configs.max_replicas_per_region = 4
+
+user root
+
+# Our setting should not affect non-constraint related zone config updates.
+statement ok
+ALTER DATABASE db CONFIGURE ZONE USING gc.ttlseconds = 1
+
+statement error pgcode 23514 constraint for "us-east-1" exceeds the configured maximum of 4 replicas
+ALTER DATABASE db CONFIGURE ZONE USING constraints = '{+region=us-east-1: 5}'
+
+statement ok
+ALTER DATABASE db CONFIGURE ZONE USING constraints = '{+region=us-east-1: 1, +region=ca-central-1: 1}', voter_constraints = '{+region=us-east-1: 3}'
+
+statement error pgcode 23514 voter constraint for "us-east-1" exceeds the configured maximum of 4 replicas
+ALTER DATABASE db CONFIGURE ZONE USING voter_constraints = '{+region=us-east-1: 5}'
+
+user host-cluster-root
+
+statement ok
+ALTER TENANT ALL SET CLUSTER SETTING sql.zone_configs.max_replicas_per_region = 3
+
+user root
+
+statement ok
+ALTER DATABASE db CONFIGURE ZONE USING gc.ttlseconds = 1
+
+statement ok
+ALTER DATABASE db CONFIGURE ZONE USING constraints = '{+region=us-east-1: 3, +region=ca-central-1: 1}', voter_constraints = '{+region=us-east-1: 2}'
+
+# Ensure that prohibited constraints don't trigger a validation error.
+statement ok
+ALTER DATABASE test CONFIGURE ZONE USING num_replicas = 4, constraints = '{-region=us-east-1: 4}'
+
+subtest end
+
+subtest privileged_range
+
+statement ok
+ALTER RANGE default CONFIGURE ZONE USING num_replicas = 10
+
+user host-cluster-root
+
+statement ok
+ALTER TENANT ALL SET CLUSTER SETTING sql.zone_configs.default_range_modifiable_by_non_root.enabled = false
+
+user root
+
+statement ok
+ALTER RANGE default CONFIGURE ZONE USING num_replicas = 10
+
+statement ok
+SET ROLE admin
+
+statement error pgcode 42501 only root users are allowed to modify the default range
+ALTER RANGE default CONFIGURE ZONE USING num_replicas = 10
+
+subtest end

pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-tenant/BUILD.bazel

@@ -9,7 +9,7 @@ go_test(
         "//pkg/ccl/logictestccl:testdata",  # keep
     ],
     exec_properties = {"test.Pool": "large"},
-    shard_count = 17,
+    shard_count = 18,
     tags = ["cpu:4"],
     deps = [
         "//pkg/base",

pkg/ccl/logictestccl/tests/multiregion-9node-3region-3azs-tenant/generated_test.go

@@ -1630,7 +1630,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/alter_table.out -
 -ERROR:  column "test" is in a primary key
 +ERROR:  column "test" is in a primary index
  alter table atacc1 drop constraint "atacc1_pkey";
-+ERROR:  relation "atacc1" (1984): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "atacc1": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  alter table atacc1 alter column test drop not null;
@@ -5404,7 +5404,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/alter_table.out -
  DELETE FROM old_system_table WHERE othercol = 'somedata';
  TRUNCATE old_system_table;
  ALTER TABLE old_system_table DROP CONSTRAINT new_system_table_pkey;
-+ERROR:  relation "old_system_table" (2082): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "old_system_table": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  ALTER TABLE old_system_table DROP COLUMN othercol;

pkg/cmd/roachtest/testdata/pg_regress/alter_table.diffs

@@ -119,7 +119,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/create_view.out -
 -ERROR:  cannot drop constraint view_base_table_pkey on table view_base_table because other objects depend on it
 -DETAIL:  view key_dependent_view depends on constraint view_base_table_pkey on table view_base_table
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "view_base_table" (325): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "view_base_table": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  CREATE VIEW key_dependent_view_no_cols AS

pkg/cmd/roachtest/testdata/pg_regress/create_view.diffs

@@ -112,7 +112,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/functional_deps.o
 -ERROR:  cannot drop constraint articles_pkey on table articles because other objects depend on it
 -DETAIL:  view fdv1 depends on constraint articles_pkey on table articles
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "articles" (1613): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  DROP VIEW fdv1;
@@ -125,14 +125,14 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/functional_deps.o
 -ERROR:  cannot drop constraint articles_pkey on table articles because other objects depend on it
 -DETAIL:  view fdv2 depends on constraint articles_pkey on table articles
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "articles" (1613): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  ALTER TABLE articles_in_category DROP CONSTRAINT articles_in_category_pkey RESTRICT; --fail
 -ERROR:  cannot drop constraint articles_in_category_pkey on table articles_in_category because other objects depend on it
 -DETAIL:  view fdv2 depends on constraint articles_in_category_pkey on table articles_in_category
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "articles_in_category" (1614): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles_in_category": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  DROP VIEW fdv2;
@@ -145,7 +145,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/functional_deps.o
 -ERROR:  cannot drop constraint articles_pkey on table articles because other objects depend on it
 -DETAIL:  view fdv3 depends on constraint articles_pkey on table articles
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "articles" (1613): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  DROP VIEW fdv3;
@@ -155,7 +155,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/functional_deps.o
 -ERROR:  cannot drop constraint articles_pkey on table articles because other objects depend on it
 -DETAIL:  view fdv4 depends on constraint articles_pkey on table articles
 -HINT:  Use DROP ... CASCADE to drop the dependent objects too.
-+ERROR:  relation "articles" (1613): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  DROP VIEW fdv4;
@@ -165,7 +165,7 @@ diff -U3 --label=/mnt/data1/postgres/src/test/regress/expected/functional_deps.o
  (0 rows)
 
  ALTER TABLE articles DROP CONSTRAINT articles_pkey RESTRICT;
-+ERROR:  relation "articles" (1613): unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
++ERROR:  relation "articles": unimplemented: primary key dropped without subsequent addition of new primary key in same transaction
 +HINT:  You have attempted to use a feature that is not yet implemented.
 +See: https://go.crdb.dev/issue-v/48026/_version_
  EXECUTE foo;  -- fail