Skip to content

Handle racey readiness probe in conduit containers #62

@lyuboxa

Description

@lyuboxa

The conduit server container will exit when the pipeline goes into degraded state (see pipelines.exit-on-degraded config option).

However, during the load up phase the probe can report ready even though conduit has not loaded the pipeline

time conduit container status
T1 Conduit boots /healthz 200 Container Ready
T2 Conduit loads pipeline /healthz 200 Container Ready
T3 Conduit exits due to pipeline failure /healthz not reachable Container Not Ready

Temporary solution may be to introduce a delay such that conduit has a chance to load the pipeline and either stabilize or exit, e.g.

Delay of t = 2

time conduit container status
T1 Conduit boots /healthz 200 Wait
T2 Conduit loads pipeline /healthz 200 Wait
T3 Conduit exits due to pipeline failure /healthz not reachable Check

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions