Skip to content

kv/kvserver: TestAdminScatterWithDrainingNodes failed #154316

@cockroach-teamcity

Description

@cockroach-teamcity

kv/kvserver.TestAdminScatterWithDrainingNodes failed on release-25.4 @ 4188e69004009122ca2c687319561e9d835027a1:

github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt(0xc02680b908, {0xda544a0, 0xc0288b4b90}, {0x0, 0x0}, {{{0xda803c0, 0xc0288b4a50}, {0x0, 0x0, 0x0}, ...}, ...}, ...)
	pkg/sql/conn_executor_prepare.go:116 +0x150
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare(0xc02680b908, {0xda544a0, 0xc0288b4b90}, {{0x0, 0x0}, {{0xda803c0, 0xc0288b4a50}, {0x0, 0x0, 0x0}, ...}, ...})
	pkg/sql/conn_executor_prepare.go:77 +0xc58
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc02680b908)
	pkg/sql/conn_executor.go:2512 +0x3447
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc02680b908, {0xda54b78, 0xc0288a6fa0}, 0xc0121236b0, 0xc028058c30, 0x0)
	pkg/sql/conn_executor.go:2288 +0x3eb
github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx.func2()
	pkg/sql/internal.go:258 +0x2b7
created by github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx in goroutine 59113
	pkg/sql/internal.go:250 +0x5ad

goroutine 58534 [select]:
github.com/cockroachdb/cockroach/pkg/util/admission.(*StoreGrantCoordinators).SetPebbleMetricsProvider.func1()
	pkg/util/admission/store_grant_coordinator.go:131 +0x19d
created by github.com/cockroachdb/cockroach/pkg/util/admission.(*StoreGrantCoordinators).SetPebbleMetricsProvider in goroutine 56005
	pkg/util/admission/store_grant_coordinator.go:121 +0x5ed

goroutine 58359 [sync.Cond.Wait, 74 minutes]:
sync.runtime_notifyListWait(0xc004c44fd0, 0x0)
	GOROOT/src/runtime/sema.go:587 +0x159
sync.(*Cond).Wait(0xc004c44fc0)
	GOROOT/src/sync/cond.go:71 +0x75
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*schedulerShard).processEvents(0xc00d0f27e0, {0xda54b78, 0xc01235c000})
	pkg/kv/kvserver/rangefeed/scheduler.go:476 +0x7bc
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Scheduler).Start.func1({0xda54b78, 0xc01235c000})
	pkg/kv/kvserver/rangefeed/scheduler.go:249 +0x1f5
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0xda54b78, 0xc01235c000})
	pkg/util/stop/stopper.go:440 +0x116
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 58434
	pkg/util/stop/stopper.go:438 +0x24f

goroutine 58512 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*mmaStoreRebalancer).run(0xc018a3f9e0, {0xda54b78, 0xc01235c000}, 0xc018fd65a0)
	pkg/kv/kvserver/mma_store_rebalancer.go:75 +0x398
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*mmaStoreRebalancer).start.func1({0xda54b78, 0xc01235c000})
	pkg/kv/kvserver/mma_store_rebalancer.go:119 +0x4d
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0xda54b78, 0xc01235c000})
	pkg/util/stop/stopper.go:440 +0x116
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 58434
	pkg/util/stop/stopper.go:438 +0x24f

goroutine 56411 [select, 74 minutes]:
github.com/cockroachdb/cockroach/pkg/sql.(*runnerCoordinator).init.func1({0xc013274480?, 0xda54b78?})
	pkg/sql/distsql_running.go:165 +0x126
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0xda54b78, 0xc00b090140})
	pkg/util/stop/stopper.go:440 +0x116
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 56418
	pkg/util/stop/stopper.go:438 +0x24f

Parameters:

  • attempt=1
  • race=true
  • run=3
  • shard=42
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-54865

Epic CRDB-55382

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.C-test-failureBroken test (automatically or manually discovered).O-robotOriginated from a bot.P-2Issues/test failures with a fix SLA of 3 monthsT-db-serverX-duplicateClosed as a duplicate of another issue.branch-release-25.4Used to mark GA and release blockers and technical advisories for 25.4

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions