Included Redpanda Console can't connect to Redpanda with activated SASL

[gabschne]

When trying to deploy a Redpanda cluster using the operator and with SASL activated, the included Redpanda console can't connect to the Redpanda cluster:

{"level":"info","ts":"2025-10-01T18:38:14.081Z","msg":"connecting to Kafka seed brokers, trying to fetch cluster metadata"}         
{"level":"error","ts":"2025-10-01T18:38:14.084Z","logger":"kafka_client","msg":"unable to initialize sasl","broker":"seed_0","err":"SASL authentication failed: security: Invalid credentials: SASL_AUTHENTICATION_FAILED: SASL Authentication failed."}                
{"level":"warn","ts":"2025-10-01T18:38:14.084Z","msg":"Failed to test Kafka connection, going to retry in 1s","remaining_retries":5}

Redpanda pod:

redpanda INFO  2025-10-01 18:38:14,083 [shard 0:kafk] security - scram_authenticator.cc:96 - Authentication failed: stored and clien
t submitted credentials do not match 

Pod template (minikube setup):

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  namespace: redpanda
  name: redpanda
spec:
  clusterSpec:
    image:
      # Tried with chart default version and newest tag 
      tag: "v25.2.7"
    tuning:
        tune_aio_events: false
    console:
      enabled: true
    tls:
      enabled: true
    auth:
      sasl:
        enabled: true
        secretRef: redpanda-superusers
        users: []
        # Have to specify the bootstrap user name, because the cluster fails to start otherwise. Might be another issue.
        bootstrapUser:
          name: bootstrap
    logging:
      usageStats:
        enabled: false
    statefulset:
      initContainerImage:
        repository: alpine
        tag: latest
      replicas: 3
      initContainers:
        setDataDirOwnership:
          enabled: true

We were able to reproduce this issue on multiple Machines and different Setups (Minikube and vanilla K8s).

Operator (chart) version: v25.1.3

[chrisseto]

Thanks for the bug report!

Unfortunately, this issue has existed for quite sometime but has gone unnoticed until recently. We're currently adding a new dedicated Console CRD which addresses this issue by using the bootstrap user.

I believe you can work around this by specifying some users under the spec.clusterSpec.auth.sasl.users. The deployment plucks off the first user from that list to gain SASL credentials.

It's may also be possible to manually specify the SASL credentials through the console stanza. I can hunt down the exact fields if that would be of interest to you.

    # Have to specify the bootstrap user name, because the cluster fails to start otherwise. Might be another issue.

That shouldn't be the case. There's a default name that's used if one isn't specified. That could be another issue or you may be "resurrecting" an existing cluster if you haven't manually removed the PVCs in between test cycles.

[gabschne]

Unfortunately, this issue has existed for quite sometime but has gone unnoticed until recently. We're currently adding a new dedicated Console CRD which addresses this issue by using the bootstrap user.

That's great to hear, thanks!

I believe you can work around this by specifying some users under the spec.clusterSpec.auth.sasl.users. The deployment plucks off the first user from that list to gain SASL credentials.

Seems like that worked! Although I'm not very happy with having to specify a superuser with a plain password in the redpanda.yaml, espacially because we're using Flux and therefore will have to commit it to our repo.

That shouldn't be the case. There's a default name that's used if one isn't specified. That could be another issue or you may be "resurrecting" an existing cluster if you haven't manually removed the PVCs in between test cycles.

Yeah, I thought so as well but was also pretty sure I deleted both, the PVCs and PVs. However, seems like Minikube is doing some advanced BTS stuff. When I recreated the whole cluster, not specifying it worked perfectly fine!