Skip to content

Experiment does not fails/breaks when k8s action cannot be performed #102

@dmartin35

Description

@dmartin35

I tried to use the terminate_pods action, but without enough permissions on my K8s cluster.
However, the experiment did go till the end with "completed" status.

Is it intended behavior when a function breaks that the CTK continues other methods ?

I would have thought to have a failure at some point

[2020-10-01 12:54:06 INFO] Action: terminate-pods
[2020-10-01 12:54:06 ERROR]   => failed: kubernetes.client.rest.ApiException: (403)
    Reason: Forbidden
    HTTP response headers: HTTPHeaderDict({'Audit-Id': '1c05c067-58fc-4d95-ae0b-240ed4c3709d', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Thu, 01 Oct 2020 12:54:06 GMT', 'Content-Length': '310'})
    HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"system:serviceaccount:chaostoolkit-run:chaostoolkit-bsm9b\" cannot list resource \"pods\" in API group \"\" in the namespace \"chaosiq-console\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
[2020-10-01 12:54:06 INFO] Pausing after activity for 2s...
[2020-10-01 12:54:11 INFO] Steady state hypothesis: Application is normal
[2020-10-01 12:54:13 INFO] Probe: application-must-respond-normally
[2020-10-01 12:54:15 INFO] Steady state hypothesis is met!
[2020-10-01 12:54:16 INFO] Let's rollback...
[2020-10-01 12:54:18 INFO] No declared rollbacks, let's move on.
[2020-10-01 12:54:19 INFO] Experiment ended with status: completed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions