Merge #154544 #154699

154544: roachtest: update DSC mixed-version test for v25.3-v25.4 r=rafiss a=rafiss

Updated the declarative schema changer job compatibility mixed-version test to test statements added to the DSC in v25.3 and v25.4, replacing the previous v24.2-v24.3 version range.

The test now validates backward and forward compatibility for:
- v25.3: ALTER TABLE ... DROP NOT NULL
- v25.4: TRUNCATE, RENAME TABLE, ALTER TABLE ... SET ON UPDATE, ALTER TABLE ... RENAME COLUMN

Added cluster version checks to conditionally execute DDLs based on the active cluster version, ensuring the test runs correctly during mixed-version upgrades.

fixes #153308
Release note: None

154699: sql: fix label for routine started metrics r=yuzefovich a=yuzefovich

During 25.4 in ef4acb4 we merged a change to add counters for stmts started and executed in routines. However, for both types we used `sql.count` label which I think is a mistake - we should use `sql.started.count` for "started" and `sql.count` for "executed" (similar to what we do for stmts outside of the routines).

Epic: CRDB-52366
Release note: None

Co-authored-by: Rafi Shamim <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>

Author: 3 people

docs/generated/metrics/metrics.yaml

@@ -664,7 +664,7 @@ layers:
       derivative: NON_NEGATIVE_DERIVATIVE
     - name: sql.routine.delete.started.count
       exported_name: sql_routine_delete_started_count
-      labeled_name: 'sql.count{query_type: routine-started-delete}'
+      labeled_name: 'sql.started.count{query_type: routine-started-delete}'
       description: Number of SQL DELETE statements started within routine invocation
       y_axis_label: SQL Statements
       type: COUNTER
@@ -674,7 +674,7 @@ layers:
       how_to_use: This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.
     - name: sql.routine.delete.started.count.internal
       exported_name: sql_routine_delete_started_count_internal
-      labeled_name: 'sql.count{query_type: routine-started-delete, query_internal: true}'
+      labeled_name: 'sql.started.count{query_type: routine-started-delete, query_internal: true}'
       description: Number of SQL DELETE statements started within routine invocation (internal queries)
       y_axis_label: SQL Internal Statements
       type: COUNTER
@@ -703,7 +703,7 @@ layers:
       derivative: NON_NEGATIVE_DERIVATIVE
     - name: sql.routine.insert.started.count
       exported_name: sql_routine_insert_started_count
-      labeled_name: 'sql.count{query_type: routine-started-insert}'
+      labeled_name: 'sql.started.count{query_type: routine-started-insert}'
       description: Number of SQL INSERT statements started within routine invocation
       y_axis_label: SQL Statements
       type: COUNTER
@@ -713,7 +713,7 @@ layers:
       how_to_use: This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.
     - name: sql.routine.insert.started.count.internal
       exported_name: sql_routine_insert_started_count_internal
-      labeled_name: 'sql.count{query_type: routine-started-insert, query_internal: true}'
+      labeled_name: 'sql.started.count{query_type: routine-started-insert, query_internal: true}'
       description: Number of SQL INSERT statements started within routine invocation (internal queries)
       y_axis_label: SQL Internal Statements
       type: COUNTER
@@ -742,7 +742,7 @@ layers:
       derivative: NON_NEGATIVE_DERIVATIVE
     - name: sql.routine.select.started.count
       exported_name: sql_routine_select_started_count
-      labeled_name: 'sql.count{query_type: routine-started-select}'
+      labeled_name: 'sql.started.count{query_type: routine-started-select}'
       description: Number of SQL SELECT statements started within routine invocation
       y_axis_label: SQL Statements
       type: COUNTER
@@ -752,7 +752,7 @@ layers:
       how_to_use: This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.
     - name: sql.routine.select.started.count.internal
       exported_name: sql_routine_select_started_count_internal
-      labeled_name: 'sql.count{query_type: routine-started-select, query_internal: true}'
+      labeled_name: 'sql.started.count{query_type: routine-started-select, query_internal: true}'
       description: Number of SQL SELECT statements started within routine invocation (internal queries)
       y_axis_label: SQL Internal Statements
       type: COUNTER
@@ -781,7 +781,7 @@ layers:
       derivative: NON_NEGATIVE_DERIVATIVE
     - name: sql.routine.update.started.count
       exported_name: sql_routine_update_started_count
-      labeled_name: 'sql.count{query_type: routine-started-update}'
+      labeled_name: 'sql.started.count{query_type: routine-started-update}'
       description: Number of SQL UPDATE statements started within routine invocation
       y_axis_label: SQL Statements
       type: COUNTER
@@ -791,7 +791,7 @@ layers:
       how_to_use: This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.
     - name: sql.routine.update.started.count.internal
       exported_name: sql_routine_update_started_count_internal
-      labeled_name: 'sql.count{query_type: routine-started-update, query_internal: true}'
+      labeled_name: 'sql.started.count{query_type: routine-started-update, query_internal: true}'
       description: Number of SQL UPDATE statements started within routine invocation (internal queries)
       y_axis_label: SQL Internal Statements
       type: COUNTER

pkg/cmd/roachtest/tests/mixed_version_job_compatibility_in_declarative_schema_changer.go

@@ -10,6 +10,7 @@ import (
 	gosql "database/sql"
 	"math/rand"
 
+	"github.com/cockroachdb/cockroach/pkg/clusterversion"
 	"github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster"
 	"github.com/cockroachdb/cockroach/pkg/cmd/roachtest/option"
 	"github.com/cockroachdb/cockroach/pkg/cmd/roachtest/registry"
@@ -27,7 +28,7 @@ func registerDeclarativeSchemaChangerJobCompatibilityInMixedVersion(r registry.R
 	// This test requires us to come back and change the stmts in executeSupportedDDLs to be those
 	// supported in the "previous" major release.
 	r.Add(registry.TestSpec{
-		Name:    "declarative_schema_changer/job-compatibility-mixed-version-V242-V243",
+		Name:    "declarative_schema_changer/job-compatibility-mixed-version-V253-V254",
 		Owner:   registry.OwnerSQLFoundations,
 		Cluster: r.MakeClusterSpec(4),
 		// Disabled on IBM because s390x is only built on master and mixed-version
@@ -107,13 +108,19 @@ func executeSupportedDDLs(
 		return err
 	}
 
-	// DDLs supported in V24_2.
-	// TODO(sql-foundations): uncomment these when the final 24.2 cluster version
-	// is created.
-	v242DDLs := []string{
-		// `ALTER DATABASE testdb CONFIGURE ZONE USING gc.ttlseconds=1000`,
-		// `ALTER TABLE testdb.testsc.t CONFIGURE ZONE USING gc.ttlseconds=2000`,
-		// `COMMENT ON TYPE testdb.testsc.typ IS 'comment'`,
+	// DDLs supported in V25_3.
+	v253DDLs := []string{
+		`ALTER TABLE testdb.testsc.t2 ALTER COLUMN i DROP NOT NULL`,
+	}
+
+	// DDLs supported in V25_4.
+	v254DDLs := []string{
+		`TRUNCATE testdb.testsc.t3`,
+		`ALTER TABLE testdb.testsc.t RENAME TO t_renamed`,
+		`ALTER TABLE testdb.testsc.t_renamed RENAME TO t`,
+		`ALTER TABLE testdb.testsc.t2 ALTER COLUMN j SET ON UPDATE j + 1`,
+		`ALTER TABLE testdb.testsc.t2 RENAME COLUMN k TO k_renamed`,
+		`ALTER TABLE testdb.testsc.t2 RENAME COLUMN k_renamed TO k`,
 	}
 
 	// Used to clean up our CREATE-d elements after we are done with them.
@@ -130,9 +137,26 @@ func executeSupportedDDLs(
 		`DROP OWNED BY foo`,
 	}
 
-	ddls := append(v242DDLs, cleanup...)
+	clusterVersion, err := helper.ClusterVersion(r)
+	if err != nil {
+		return err
+	}
+	if clusterVersion.AtLeast(clusterversion.V25_3.Version()) {
+		for _, ddl := range v253DDLs {
+			if err := helper.ExecWithGateway(r, nodes, ddl); err != nil {
+				return err
+			}
+		}
+	}
+	if clusterVersion.AtLeast(clusterversion.V25_4.Version()) {
+		for _, ddl := range v254DDLs {
+			if err := helper.ExecWithGateway(r, nodes, ddl); err != nil {
+				return err
+			}
+		}
+	}
 
-	for _, ddl := range ddls {
+	for _, ddl := range cleanup {
 		if err := helper.ExecWithGateway(r, nodes, ddl); err != nil {
 			return err
 		}

pkg/sql/exec_util.go

@@ -1132,7 +1132,7 @@ var (
 		Help:         "Number of SQL SELECT statements started within routine invocation",
 		Measurement:  "SQL Statements",
 		Unit:         metric.Unit_COUNT,
-		LabeledName:  "sql.count",
+		LabeledName:  "sql.started.count",
 		StaticLabels: metric.MakeLabelPairs(metric.LabelQueryType, "routine-started-select"),
 		Category:     metric.Metadata_SQL,
 		HowToUse:     "This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.",
@@ -1142,7 +1142,7 @@ var (
 		Help:         "Number of SQL UPDATE statements started within routine invocation",
 		Measurement:  "SQL Statements",
 		Unit:         metric.Unit_COUNT,
-		LabeledName:  "sql.count",
+		LabeledName:  "sql.started.count",
 		StaticLabels: metric.MakeLabelPairs(metric.LabelQueryType, "routine-started-update"),
 		Category:     metric.Metadata_SQL,
 		HowToUse:     "This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.",
@@ -1152,7 +1152,7 @@ var (
 		Help:         "Number of SQL INSERT statements started within routine invocation",
 		Measurement:  "SQL Statements",
 		Unit:         metric.Unit_COUNT,
-		LabeledName:  "sql.count",
+		LabeledName:  "sql.started.count",
 		StaticLabels: metric.MakeLabelPairs(metric.LabelQueryType, "routine-started-insert"),
 		Category:     metric.Metadata_SQL,
 		HowToUse:     "This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.",
@@ -1162,7 +1162,7 @@ var (
 		Help:         "Number of SQL DELETE statements started within routine invocation",
 		Measurement:  "SQL Statements",
 		Unit:         metric.Unit_COUNT,
-		LabeledName:  "sql.count",
+		LabeledName:  "sql.started.count",
 		StaticLabels: metric.MakeLabelPairs(metric.LabelQueryType, "routine-started-delete"),
 		Category:     metric.Metadata_SQL,
 		HowToUse:     "This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the SQL Activity pages to investigate interesting outliers or patterns. For example, on the Transactions page and the Statements page, sort on the Execution Count column. To find problematic sessions, on the Sessions page, sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application.",