Merge #147089 #147439 #147447 #147635

147089: db-console: add client side hot range filtering by node r=angles-n-daemons a=angles-n-daemons

db-console: add client side hot range filtering by node

Loading the hot ranges page is a performance intensive operation in large clusters, due to the fact the a cluster fanout is required to gather the information. Despite having a node filter within the component, the filtering has historically been applied after the api has responded with the full cluster's information.

This change moves the filtering of the nodes to the back end, and cleans up some of how the filters are organized. It also merges the uses of the NodeRegionSelector so that it can be shared between the Databases and Hot Ranges pages.

Fixes: #143528
Epic: CRDB-43150

Release note (ui change): Moves the hot ranges node filter out of the primary filter container in the hot ranges page, and applies the filtering on the backend.

147439: ui: surface SQL commenter query tags in insights r=angles-n-daemons a=angles-n-daemons

This commit adds support for displaying SQL commenter query tags in the CockroachDB DB Console insights UI, addressing GitHub issue #146664.

Changes:
- Add query_tags to statement insights API response
- Update TypeScript types to include queryTags field
- Add Query Tags column to statement insights table (hidden by default)
- Display query tags on statement insight detail pages
- Add proper column titles and tooltips for query tags

The query tags were already being stored in the backend execution insights tables by PR #145435. This change surfaces that data in the frontend UI, allowing users to correlate query performance with application context provided via SQL commenter tags.

Fixes #146664

Release note (ui change): The DB Console insights page now displays SQL commenter query tags for statement executions. Query tags provide application context (such as application name, user ID, or feature flags) embedded in SQL comments using the sqlcommenter format. This information can help correlate slow query performance with specific application state. The Query Tags column is available in the statement insights table but hidden by default - it can be enabled via the Columns selector.

🤖 Generated with [Claude Code](https://claude.ai/code)

147447: ui: fix statement activity timepicker for sub-hour ranges r=angles-n-daemons a=angles-n-daemons

ui: fix statement activity timepicker for sub-hour ranges

This commit fixes a bug where the statement activity time picker wasn't working correctly with time ranges less than an hour when sql.stats.aggregation.interval is set to values smaller than 1h.

The issue was that API calls were using toRoundedDateRange() which rounds timestamps to hour boundaries instead of toDateRange() which uses the exact selected time range. This prevented sub-hour time ranges from working properly even when the aggregation interval supported them.

Changed the following components to use toDateRange():
- StatementsPage
- TransactionsPage
- TransactionDetails
- StatementDetails selectors
- IndexDetails API

Fixes #145430

Epic: None
Release note (bug fix): Fixed statement activity page time picker to work correctly with time ranges less than an hour when sql.stats.aggregation.interval is configured to sub-hour values. Previously, selecting a 10-minute window would query for a full hour of data instead of the precise selected range.

147635: roachtest: fail fixture roachtests on failed backups r=msbutler a=kev-cao

Previously, if a backup job failed during the fixture roachtest, the test would continue along until the required number of backups completed. In this situation, if a backup failed for whatever reason, we would ignore it. This commit teaches the fixture roachtest to detect when a backup job has failed and to fail with the corresponding error.

Epic: None

Release note: None

Co-authored-by: Brian Dillmann <[email protected]>
Co-authored-by: Kevin Cao <[email protected]>

Author: 3 people

pkg/cmd/roachtest/tests/backup_fixtures.go

@@ -257,7 +257,7 @@ func (bd *backupDriver) monitorBackups(ctx context.Context) error {
 	const (
 		WaitingFirstFull = iota
 		RunningIncrementals
-		WaitingCompactions
+		WaitingCompletion
 		Done
 	)
 	state := WaitingFirstFull
@@ -267,30 +267,41 @@ func (bd *backupDriver) monitorBackups(ctx context.Context) error {
 		if err != nil {
 			return err
 		}
+		_, backupRunning, backupFailed, err := bd.backupJobStates(sql)
+		if err != nil {
+			return err
+		}
 		switch state {
 		case WaitingFirstFull:
 			var activeScheduleCount int
 			scheduleCountQuery := fmt.Sprintf(
 				`SELECT count(*) FROM [SHOW SCHEDULES] WHERE label='%s' AND schedule_status='ACTIVE'`, scheduleLabel,
 			)
 			sql.QueryRow(bd.t, scheduleCountQuery).Scan(&activeScheduleCount)
-			if activeScheduleCount < 2 {
+			if len(backupFailed) > 0 {
+				return errors.Newf("backup jobs failed while waiting first full: %v", backupFailed)
+			} else if activeScheduleCount < 2 {
 				bd.t.L().Printf(`First full backup still running`)
 			} else {
 				state = RunningIncrementals
 			}
 		case RunningIncrementals:
 			var backupCount int
+			// We track completed backups via SHOW BACKUP as opposed to SHOW JOBs in
+			// the case that a fixture runs for a long enough time that old backup
+			// jobs stop showing up in SHOW JOBS.
 			backupCountQuery := fmt.Sprintf(
 				`SELECT count(DISTINCT end_time) FROM [SHOW BACKUP FROM LATEST IN '%s']`, fixtureURI.String(),
 			)
 			sql.QueryRow(bd.t, backupCountQuery).Scan(&backupCount)
 			bd.t.L().Printf(`%d scheduled backups taken`, backupCount)
 
-			if bd.sp.fixture.CompactionThreshold > 0 {
+			if len(backupFailed) > 0 {
+				return errors.Newf("backup jobs failed while running incrementals: %v", backupFailed)
+			} else if bd.sp.fixture.CompactionThreshold > 0 {
 				bd.t.L().Printf("%d compaction jobs succeeded, %d running", len(compSuccess), len(compRunning))
 				if len(compFailed) > 0 {
-					return errors.Newf("compaction jobs failed: %v", compFailed)
+					return errors.Newf("compaction jobs failed while running incrementals: %v", compFailed)
 				}
 			}
 
@@ -299,15 +310,19 @@ func (bd *backupDriver) monitorBackups(ctx context.Context) error {
 					`PAUSE SCHEDULES WITH x AS (SHOW SCHEDULES) SELECT id FROM x WHERE label = '%s'`, scheduleLabel,
 				)
 				sql.Exec(bd.t, pauseSchedulesQuery)
-				if len(compRunning) > 0 {
-					state = WaitingCompactions
+				if len(compRunning) > 0 || len(backupRunning) > 0 {
+					state = WaitingCompletion
 				} else {
 					state = Done
 				}
 			}
-		case WaitingCompactions:
-			if len(compFailed) > 0 {
-				return errors.Newf("compaction jobs failed: %v", compFailed)
+		case WaitingCompletion:
+			if len(backupFailed) > 0 {
+				return errors.Newf("backup jobs failed while waiting completion: %v", backupFailed)
+			} else if len(compFailed) > 0 {
+				return errors.Newf("compaction jobs failed while waiting completion: %v", compFailed)
+			} else if len(backupRunning) > 0 {
+				bd.t.L().Printf("waiting for %d backup jobs to finish", len(backupRunning))
 			} else if len(compRunning) > 0 {
 				bd.t.L().Printf("waiting for %d compaction jobs to finish", len(compRunning))
 			} else {
@@ -332,20 +347,45 @@ func (bd *backupDriver) compactionJobStates(
 	if bd.sp.fixture.CompactionThreshold == 0 {
 		return nil, nil, nil, nil
 	}
-	compactionQuery := `SELECT job_id, status, error FROM [SHOW JOBS] WHERE job_type = 'BACKUP' AND
-	description ILIKE 'COMPACT BACKUPS%'`
-	rows := sql.Query(bd.t, compactionQuery)
+	s, r, f, err := bd.queryJobStates(
+		sql, "job_type = 'BACKUP' AND description ILIKE 'COMPACT BACKUPS%'",
+	)
+	return s, r, f, errors.Wrapf(err, "error querying compaction job states")
+}
+
+// backupJobStates returns the state of the backup jobs, returning
+// a partition of jobs that succeeded, are running, and failed.
+func (bd *backupDriver) backupJobStates(
+	sql *sqlutils.SQLRunner,
+) ([]jobMeta, []jobMeta, []jobMeta, error) {
+	s, r, f, err := bd.queryJobStates(
+		sql, "job_type = 'BACKUP' AND description ILIKE 'BACKUP %'",
+	)
+	return s, r, f, errors.Wrapf(err, "error querying backup job states")
+}
+
+// queryJobStates queries the job table and returns a partition of jobs that
+// succeeded, are running, and failed. The filter is applied to the query to
+// limit the jobs searched. If the filter is empty, all jobs are searched.
+func (bd *backupDriver) queryJobStates(
+	sql *sqlutils.SQLRunner, filter string,
+) ([]jobMeta, []jobMeta, []jobMeta, error) {
+	query := "SELECT job_id, status, error FROM [SHOW JOBS]"
+	if filter != "" {
+		query += fmt.Sprintf(" WHERE %s", filter)
+	}
+	rows := sql.Query(bd.t, query)
 	defer rows.Close()
-	var compJobs []jobMeta
+	var jobMetas []jobMeta
 	for rows.Next() {
 		var job jobMeta
 		if err := rows.Scan(&job.jobID, &job.state, &job.error); err != nil {
-			return nil, nil, nil, errors.Wrapf(err, "error scanning compaction job")
+			return nil, nil, nil, errors.Wrapf(err, "error scanning job")
 		}
-		compJobs = append(compJobs, job)
+		jobMetas = append(jobMetas, job)
 	}
 	var successes, running, failures []jobMeta
-	for _, job := range compJobs {
+	for _, job := range jobMetas {
 		switch job.state {
 		case jobs.StateSucceeded:
 			successes = append(successes, job)
@@ -354,7 +394,7 @@ func (bd *backupDriver) compactionJobStates(
 		case jobs.StateFailed:
 			failures = append(failures, job)
 		default:
-			bd.t.L().Printf(`unexpected compaction job %d in state %s`, job.jobID, job.state)
+			bd.t.L().Printf(`unexpected job %d in state %s`, job.jobID, job.state)
 		}
 	}
 	return successes, running, failures, nil

pkg/ui/workspaces/cluster-ui/src/api/indexDetailsApi.ts

@@ -20,7 +20,7 @@ import {
 
 import { INTERNAL_APP_NAME_PREFIX } from "../activeExecutions/activeStatementUtils";
 import { AggregateStatistics } from "../statementsTable";
-import { TimeScale, toRoundedDateRange } from "../timeScaleDropdown";
+import { TimeScale, toDateRange } from "../timeScaleDropdown";
 
 export type TableIndexStatsRequest =
   cockroach.server.serverpb.TableIndexStatsRequest;
@@ -77,7 +77,7 @@ export function StatementsListRequestFromDetails(
   ts: TimeScale,
 ): StatementsUsingIndexRequest {
   if (ts === null) return { table, index, database };
-  const [start, end] = toRoundedDateRange(ts);
+  const [start, end] = toDateRange(ts);
   return { table, index, database, start, end };
 }
 

pkg/ui/workspaces/cluster-ui/src/api/nodesApi.ts

@@ -21,6 +21,7 @@ export const getNodes =
   };
 
 export type NodeStatus = {
+  id: NodeID;
   region: string;
   stores: StoreID[];
 };
@@ -48,7 +49,9 @@ export const useNodeStatuses = () => {
           .node_id as NodeID;
       });
 
-      nodeStatusByID[ns.desc.node_id as NodeID] = {
+      const id = ns.desc.node_id as NodeID;
+      nodeStatusByID[id] = {
+        id,
         region: getRegionFromLocality(ns.desc.locality),
         stores: ns.store_statuses?.map(s => s.desc.store_id as StoreID),
       };

pkg/ui/workspaces/cluster-ui/src/api/stmtInsightsApi.ts

@@ -66,6 +66,7 @@ export type StmtInsightsResponseRow = {
   error_code: string;
   last_error_redactable: string;
   status: StatementStatus;
+  query_tags: Array<{ name: string; value: string }>;
 };
 
 const stmtColumns = `
@@ -97,7 +98,8 @@ plan_gist,
 cpu_sql_nanos,
 error_code,
 last_error_redactable,
-status
+status,
+query_tags
 `;
 
 const stmtInsightsOverviewQuery = (req?: StmtInsightsReq): string => {
@@ -241,6 +243,7 @@ export function formatStmtInsights(
       errorCode: row.error_code,
       errorMsg: row.last_error_redactable,
       status: row.status,
+      queryTags: row.query_tags || [],
     } as StmtInsightEvent;
   });
 }

pkg/ui/workspaces/cluster-ui/src/components/index.ts

@@ -0,0 +1,11 @@
+// Copyright 2021 The Cockroach Authors.
+//
+// Use of this software is governed by the CockroachDB Software License
+// included in the /LICENSE file.
+
+export * from "./links/indexStatsLink";
+export * from "./liveDataPercent/liveDataPercent";
+export * from "./nodeSelector/nodeSelector";
+export * from "./tooltip";
+// Seems like there are duplicate exports with this file. We omit it here.
+// export * from "./tooltipMessages";

pkg/ui/workspaces/cluster-ui/src/components/nodeSelector/nodeSelector.module.scss

@@ -0,0 +1,50 @@
+// Copyright 2025 The Cockroach Authors.
+//
+// Use of this software is governed by the CockroachDB Software License
+// included in the /LICENSE file.
+
+@import "src/core/index.module.scss";
+
+.selector {
+  min-width: 200px;
+}
+
+.regionsSection {
+  max-height: 40vh;
+  overflow-y: scroll;
+}
+
+.option {
+  padding: 2px 8px;
+  cursor: pointer;
+  input {
+    margin-right: 4px;
+  }
+
+  label {
+    cursor: pointer; /* Explicitly set cursor for labels */
+    margin-left: 4px;
+    display: inline-block;
+  }
+}
+
+.regionNodeOption {
+  padding-left: 24px;
+}
+.divider {
+  margin: 4px 0;
+}
+
+.applyBtnContainer {
+  padding: 4px 12px 16px 12px;
+}
+
+.applyBtn {
+  width: 100%;
+
+  div {
+    div {
+      justify-content: center;
+    }
+  }
+}