Skip to content

Concerned about healthcheck condition checks #169

@iciclespider

Description

@iciclespider

What happened?

I examined the health checks and noticed some possible problems with the way conditions are checked. Many of the checks loop through the conditions array checking for both valid and invalid conditions and returns the state of the first condition it encounters. If a resource has both valid and invalid conditions in place, the success of the health check will be determined by the order of the conditions. The same exact set of conditions, but in a different order, can return different results.

For example this check in the HPA check:

	for _, condition := range hpa.Status.Conditions {
		// Check for degraded conditions
		switch condition.Type {
		case "FailedGetScale", "FailedUpdateScale", "FailedGetResourceMetric", "InvalidSelector":
			if condition.Status == "True" {
				return false
			}
		}

		// Check for healthy conditions
		switch condition.Type {
		case autoscalingv2.ScalingActive:
			if condition.Status == "True" {
				return true
			}
		case autoscalingv2.ScalingLimited:
			if condition.Status == "True" {
				return true
			}
		}
	}

The conditions array is sequentially walked through, and if both valid and invalid conditions are in place, the success of the check is determined by the order of the conditions.

What environment did it happen in?

v0.6.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions