Skip to content

e2e tests fail #2030

New issue
New issue
Open
Open
e2e tests fail#2030
@stzov

Description

@stzov
stzov
opened on Sep 25, 2025

What steps did you take and what happened:
I initially tried the default
sonobuoy run --wait
and the tests failed, and then I tried
sonobuoy run --mode quick --skip-preflight --security-context-mode none --wait
and that failed too.
In both cases I got

$ sonobuoy status
         PLUGIN     STATUS   RESULT   COUNT                                PROGRESS
            e2e   complete   failed       1   Passed:  0, Failed:  0, Remaining:  1
   systemd-logs   complete   passed       4                                        

Sonobuoy has completed. Use `sonobuoy retrieve` to get results.

It seems as if none of the e2e tests run, quick mode or not.
The result is always failed and both passed and failed count columns 0.

What did you expect to happen:
Best case the test(s) to pass, worst case the test result to be failed but also at least one count for the failed column with an explanation somewhere.
Forgive me if I'm missing something but I don't getting any lead with this fails.

Anything else you would like to add:
I don't see any error in the pod logs

time="2025-09-24T12:10:41Z" level=info msg="Scanning plugins in ./plugins.d (pwd: /)"
time="2025-09-24T12:10:41Z" level=info msg="Scanning plugins in /etc/sonobuoy/plugins.d (pwd: /)"
time="2025-09-24T12:10:41Z" level=info msg="Directory (/etc/sonobuoy/plugins.d) does not exist"
time="2025-09-24T12:10:41Z" level=info msg="Scanning plugins in ~/sonobuoy/plugins.d (pwd: /)"
time="2025-09-24T12:10:41Z" level=info msg="Directory (~/sonobuoy/plugins.d) does not exist"
time="2025-09-24T12:10:42Z" level=info msg="Starting server Expected Results: [{global e2e 0} {aks-systempoolv3-18198765-vmss00000c systemd-logs 0} {aks-systempoolv3-18198765-vmss00000f systemd-logs 0} {aks-userpoolv4-18198765-vmss000000 systemd-logs 0} {aks-userpoolv5-18198765-vmss000000 systemd-logs 0}]"
time="2025-09-24T12:10:42Z" level=info msg="Starting annotation update routine"
time="2025-09-24T12:10:42Z" level=info msg="Starting aggregation server" address=0.0.0.0 port=8080
time="2025-09-24T12:10:42Z" level=info msg="Received plugin launch order:"
time="2025-09-24T12:10:42Z" level=info msg="e2e: 0"
time="2025-09-24T12:10:42Z" level=info msg="systemd-logs: 0"
time="2025-09-24T12:10:42Z" level=info msg="Running plugin" plugin=e2e
time="2025-09-24T12:10:42Z" level=info msg="Running plugin" plugin=systemd-logs
time="2025-09-24T12:10:42Z" level=info msg="Launching plugin systemd-logs with order 0"
time="2025-09-24T12:10:42Z" level=info msg="Launching plugin e2e with order 0"
time="2025-09-24T12:10:43Z" level=info msg="received request" client_cert="[e2e]" method=POST plugin_name=e2e url=/api/v1/progress/global/e2e
time="2025-09-24T12:10:43Z" level=info msg="received request" client_cert="[e2e]" method=POST plugin_name=e2e url=/api/v1/progress/global/e2e
time="2025-09-24T12:10:49Z" level=info msg="received request" client_cert="[systemd-logs]" method=PUT node=aks-systempoolv3-18198765-vmss00000f plugin_name=systemd-logs url=/api/v1/results/by-node/aks-systempoolv3-18198765-vmss00000f/systemd-logs
time="2025-09-24T12:10:49Z" level=info msg="received request" client_cert="[systemd-logs]" method=PUT node=aks-userpoolv5-18198765-vmss000000 plugin_name=systemd-logs url=/api/v1/results/by-node/aks-userpoolv5-18198765-vmss000000/systemd-logs
time="2025-09-24T12:10:49Z" level=info msg="received request" client_cert="[systemd-logs]" method=PUT node=aks-userpoolv4-18198765-vmss000000 plugin_name=systemd-logs url=/api/v1/results/by-node/aks-userpoolv4-18198765-vmss000000/systemd-logs
time="2025-09-24T12:10:50Z" level=info msg="received request" client_cert="[systemd-logs]" method=PUT node=aks-systempoolv3-18198765-vmss00000c plugin_name=systemd-logs url=/api/v1/results/by-node/aks-systempoolv3-18198765-vmss00000c/systemd-logs
time="2025-09-24T12:40:43Z" level=info msg="received request" client_cert="[e2e]" method=POST plugin_name=e2e url=/api/v1/progress/global/e2e
time="2025-09-24T12:40:49Z" level=info msg="received request" client_cert="[e2e]" method=PUT plugin_name=e2e url=/api/v1/results/global/e2e
time="2025-09-24T12:40:50Z" level=info msg="All plugins have completed, status has been updated"
time="2025-09-24T12:40:52Z" level=info msg="Shutting down aggregation server"
time="2025-09-24T12:40:52Z" level=info msg="Resources is not set explicitly implying query all resources, but skipping secrets for safety. Specify the value explicitly in Resources to gather this data."
time="2025-09-24T12:40:52Z" level=info msg="Collecting Node Configuration and Health..."
time="2025-09-24T12:40:52Z" level=info msg="Creating host results for aks-systempoolv3-18198765-vmss00000c under /tmp/sonobuoy/aea95a8c-b899-48da-abbb-c18ca548dff8/hosts/aks-systempoolv3-18198765-vmss00000c\n"
time="2025-09-24T12:40:52Z" level=info msg="Creating host results for aks-systempoolv3-18198765-vmss00000f under /tmp/sonobuoy/aea95a8c-b899-48da-abbb-c18ca548dff8/hosts/aks-systempoolv3-18198765-vmss00000f\n"
time="2025-09-24T12:40:52Z" level=info msg="Creating host results for aks-userpoolv4-18198765-vmss000000 under /tmp/sonobuoy/aea95a8c-b899-48da-abbb-c18ca548dff8/hosts/aks-userpoolv4-18198765-vmss000000\n"
time="2025-09-24T12:40:52Z" level=info msg="Creating host results for aks-userpoolv5-18198765-vmss000000 under /tmp/sonobuoy/aea95a8c-b899-48da-abbb-c18ca548dff8/hosts/aks-userpoolv5-18198765-vmss000000\n"
time="2025-09-24T12:40:52Z" level=info msg="Running cluster queries"
W0924 12:40:53.334452       1 warnings.go:70] v1 ComponentStatus is deprecated in v1.19+
time="2025-09-24T12:40:54Z" level=info msg="Querying server version and API Groups"
time="2025-09-24T12:40:54Z" level=info msg="Filtering namespaces based on the following regex:.*"
time="2025-09-24T12:40:54Z" level=info msg="Namespace calico-system Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace capsule-system Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace cert-manager Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace default Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace external-dns Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace gatekeeper-system Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace keda Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace kube-node-lease Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace kube-public Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace kube-system Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace linkerd Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace linkerd-viz Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace linkerd-viz-policy-exceptions Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace sonobuoy Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace tigera-operator Matched=true"
time="2025-09-24T12:40:54Z" level=info msg="Namespace vault Matched=true"
time="2025-09-24T12:40:56Z" level=info msg="Running ns query (calico-system)"
time="2025-09-24T12:40:59Z" level=info msg="Running ns query (capsule-system)"
time="2025-09-24T12:41:01Z" level=info msg="Running ns query (cert-manager)"
time="2025-09-24T12:41:04Z" level=info msg="Running ns query (default)"
time="2025-09-24T12:41:25Z" level=info msg="Running ns query (external-dns)"
time="2025-09-24T12:41:28Z" level=info msg="Running ns query (gatekeeper-system)"
time="2025-09-24T12:41:33Z" level=info msg="Running ns query (keda)"
time="2025-09-24T12:41:35Z" level=info msg="Running ns query (kube-node-lease)"
time="2025-09-24T12:41:38Z" level=info msg="Running ns query (kube-public)"
time="2025-09-24T12:41:41Z" level=info msg="Running ns query (kube-system)"
time="2025-09-24T12:41:43Z" level=info msg="Running ns query (linkerd)"
time="2025-09-24T12:41:46Z" level=info msg="Running ns query (linkerd-viz)"
time="2025-09-24T12:41:49Z" level=info msg="Running ns query (linkerd-viz-policy-exceptions)"
time="2025-09-24T12:41:51Z" level=info msg="Running ns query (sonobuoy)"
time="2025-09-24T12:41:54Z" level=info msg="Running ns query (tigera-operator)"
time="2025-09-24T12:41:57Z" level=info msg="Running ns query (vault)"
time="2025-09-24T12:42:02Z" level=info msg="Namespace calico-system Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace capsule-system Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace cert-manager Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace default Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace external-dns Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace gatekeeper-system Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace keda Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace kube-node-lease Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace kube-public Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace kube-system Matched=true"
time="2025-09-24T12:42:02Z" level=info msg="Namespace linkerd Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace linkerd-viz Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace linkerd-viz-policy-exceptions Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace sonobuoy Matched=true"
time="2025-09-24T12:42:02Z" level=info msg="Namespace tigera-operator Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="Namespace vault Matched=false"
time="2025-09-24T12:42:02Z" level=info msg="querying pod logs under namespace kube-system"
time="2025-09-24T12:42:02Z" level=info msg="Collecting Pod Logs by namespace (kube-system)"
time="2025-09-24T12:42:22Z" level=info msg="querying pod logs under namespace sonobuoy"
time="2025-09-24T12:42:22Z" level=info msg="Collecting Pod Logs by namespace (sonobuoy)"
time="2025-09-24T12:42:22Z" level=info msg="Collecting Pod Logs by FieldSelectors: []"
time="2025-09-24T12:42:22Z" level=info msg="recording query times at /tmp/sonobuoy/aea95a8c-b899-48da-abbb-c18ca548dff8/meta/query-time.json"
time="2025-09-24T12:42:22Z" level=info msg="Log lines after this point will not appear in the downloaded tarball."
time="2025-09-24T12:42:22Z" level=info msg="Invoking plugin cleanup" plugin=e2e
time="2025-09-24T12:42:22Z" level=info msg="Invoking plugin cleanup" plugin=systemd-logs
time="2025-09-24T12:42:39Z" level=info msg="Results available at /tmp/sonobuoy/202509241210_sonobuoy_aea95a8c-b899-48da-abbb-c18ca548dff8.tar.gz"
time="2025-09-24T12:42:39Z" level=info msg="no-exit was specified, sonobuoy is now blocking"

Environment:

  • Sonobuoy version: v0.57.3
  • Kubernetes version: 1.31
  • Cloud provider or hardware configuration: AKS

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions