-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Feature Description
Emit a metric, available in Prometheus, like vttablet_tablet_type
metric to indicate if that tablet target type is currently primary, replica, etc. Alternately include tablet type in existing status metrics like vttablet_tablet_state
.
Use Case(s)
Use case is to detect when the primary is not in serving state. We have vttablet_tablet_state
metric for that, but it does not include the tablet type.
The vttablet_tablet_type_count
metric doesn't tell us what the current target type of the tablet is, so it doesn't help very much with this use case. The TabletType
stat does have what we want, but it is not exported to Prometheus.
Metrics recently added by @rafer can very nearly be used to satisfy this use case: #13521. However, those metrics are not updated when there are no queries, nor in the case of FailedPrecondition
errors like wrong tablet type
:
vitess/go/vt/vttablet/tabletserver/state_manager.go
Lines 386 to 400 in a5e819d
if sm.state != StateServing || !sm.replHealthy { | |
// This specific error string needs to be returned for vtgate buffering to work. | |
return vterrors.New(vtrpcpb.Code_CLUSTER_EVENT, vterrors.NotServing) | |
} | |
shuttingDown := sm.wantState != StateServing | |
if shuttingDown && !allowOnShutdown { | |
// This specific error string needs to be returned for vtgate buffering to work. | |
return vterrors.New(vtrpcpb.Code_CLUSTER_EVENT, vterrors.ShuttingDown) | |
} | |
err = sm.verifyTargetLocked(ctx, target) | |
if err != nil { | |
return err | |
} |