This server is not the leader for that topic-partition .

[L040639]

{"level":"ERROR","timestamp":"2025-09-03T17:42:32.658Z","logger":"kafkajs","message":"[Connection] Response ListOffsets(key: 2, version: 3)","broker":"XXXXX","clientId":"XXXXX","error":"This server is not the leader for that topic-partition","correlationId":651829,"size":113}
{"level":"ERROR","timestamp":"2025-09-03T17:42:32.658Z","logger":"kafkajs","message":"[Connection] Response ListOffsets(key: 2, version: 3)","broker":"XXXXX","clientId":"XXXXX","error":"This server is not the leader for that topic-partition","correlationId":651829,"size":113}

we are facing above client side issue with one topic in the production we have 3 brokers and the issue topic has 3 replicas and 1 partiton , so every broker should has already a replica of the topic and the topic was deployed and configured correctly . producing and consuming is working for other topic not only one topic .

below are some broker logs getting continuously i find, but these are not errors. Need some help here .

2025-09-03 16:50:59,746 INFO [GroupCoordinator 0]: Assignment received from leader f01b6126-2681-41ce-a48d-31af52c25d3b-84f9fc9e-95a0-41c9-873f-3c8737f9ebdc for group XXXX for generation 783447. The group has 2 members, 0 of which are static. (kafka.coordinator.group.GroupCoordinator) [data-plane-kafka-request-handler-4]
2025-09-03 16:50:59,753 INFO [GroupCoordinator 0]: Preparing to rebalance group XXXX in state PreparingRebalance with old generation 783447 (__consumer_offsets-4) (reason: Leader f01b6126-2681-41ce-a48d-31af52c25d3b-84f9fc9e-95a0-41c9-873f-3c8737f9ebdc re-joining group during Stable; client reason: not provided) (kafka.coordinator.group.GroupCoordinator) [data-plane-kafka-request-handler-1]
2025-09-03 16:50:59,757 INFO [GroupCoordinator 0]: Stabilized group f01b6126-2681-41ce-a48d-31af52c25d3b.consent-listener-vv generation 783448 (__consumer_offsets-4) with 2 members (kafka.coordinator.group.GroupCoordinator) [data-plane-kafka-request-handler-0]

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  namespace: <XXX>
spec:
  kafka:
    authorization:
      type: opa
      url: https://<XXX>/v1/data/kafka/allow
      allowOnError: false
      enableMetrics: true
      tlsTrustedCertificates:
        - secretName: lk-ca-root-bundle
          certificate: ca.crt
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: '3.9'
      connections.max.idle.ms: 1800000
    storage:
      type: persistent-claim
      size: 500Gi
      class: gp3
    listeners:
      - authentication:
          checkAccessTokenType: false
          clientId: <XX>
          clientSecret:
            key: secret
            secretName: entra-id-client
          jwksEndpointUri: '<XX>'
          maxSecondsWithoutReauthentication: 3600
          enableMetrics: true
          type: oauth
          validIssuerUri: '<XXX>'
          userNameClaim: azp
        configuration:
          brokerCertChainAndKey:
            certificate: tls.crt
            key: tls.key
            secretName: kafka-streams-prd-cert-secret
        name: oauth
        port: 9093
        tls: true
        type: internal
      - authentication:
          checkAccessTokenType: false
          clientId: <XXX>
          clientSecret:
            key: secret
            secretName: entra-id-client
          jwksEndpointUri: '<XXX>'
          maxSecondsWithoutReauthentication: 3600
          enableMetrics: true
          type: oauth
          validIssuerUri: '<XXX>'
          userNameClaim: azp
        configuration:
          brokerCertChainAndKey:
            certificate: tls.crt
            key: tls.key
            secretName: kafka-streams-prd-cert-secret
          bootstrap:
            alternativeNames:
            - <XX>
            annotations:
              external-dns.alpha.kubernetes.io/hostname: <XX>.
              external-dns.alpha.kubernetes.io/access: "private"
          brokers:
          - broker: 0
            advertisedHost: <XXX>
            annotations:
              external-dns.alpha.kubernetes.io/hostname: <XXXX>.
              external-dns.alpha.kubernetes.io/access: "private"
          - broker: 1
            advertisedHost: <XXX>
            annotations:
              external-dns.alpha.kubernetes.io/hostname: <XXX>.
              external-dns.alpha.kubernetes.io/access: "private"
          - broker: 2
            advertisedHost: <XXX>
            annotations:
              external-dns.alpha.kubernetes.io/hostname: <XXX>.
              external-dns.alpha.kubernetes.io/access: "private"
        name: external
        port: 9094
        tls: true
        type: loadbalancer
    version: 3.9.0
    replicas: 3
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          key: kafka-metrics-config.yml
          name: kafka-streams-prd-metrics
    template:
      externalBootstrapService:
        metadata:
          annotations:
            service.beta.kubernetes.io/aws-load-balancer-type: nlb
            service.beta.kubernetes.io/aws-load-balancer-scheme: internal
            service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "true"
            service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"
            service.beta.kubernetes.io/aws-load-balancer-listener-attributes.TCP-9094: tcp.idle_timeout.seconds=5000
      perPodService:
        metadata:
          annotations:
            service.beta.kubernetes.io/aws-load-balancer-type: nlb
            service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
            service.beta.kubernetes.io/aws-load-balancer-scheme: internal
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "true"
            service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"
            service.beta.kubernetes.io/aws-load-balancer-listener-attributes.TCP-9094: tcp.idle_timeout.seconds=5000
      pod:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: <XX>   ##dedicated nodegroup for kafka
                      operator: Exists
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: strimzi.io/cluster
                      operator: In
                      values:
                        - kafka-cluster
                    - key: strimzi.io/component-type
                      operator: In
                      values:
                        - kafka
                topologyKey: kubernetes.io/hostname
        tolerations:
          - key: "<XX>"
            operator: "Exists"
            effect: "NoSchedule"
        topologySpreadConstraints:
          - maxSkew: 1
            topologyKey: topology.kubernetes.io/zone
            whenUnsatisfiable: ScheduleAnyway
            labelSelector:
              matchLabels:
                strimzi.io/cluster: kafka-cluster
                strimzi.io/component-type: kafka
  entityOperator:
    topicOperator: {}
    userOperator: {}
    template:
      pod:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: <XXX>
                      operator: Exists
        tolerations:
          - key: "<XX>"
            operator: "Exists"
            effect: "NoSchedule"
  zookeeper:
    storage:
      type: persistent-claim
      class: gp3
      size: 50Gi
    replicas: 3
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-streams-prd-metrics
          key: zookeeper-metrics-config.yml
    template:
      pod:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: <XX>  ##dedicated nodegroup for kafka
                      operator: Exists
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                    - key: strimzi.io/cluster
                      operator: In
                      values:
                        - kafka-cluster
                    - key: strimzi.io/component-type
                      operator: In
                      values:
                        - zookeeper
                topologyKey: kubernetes.io/hostname
        tolerations:
          - key: "<XX>"
            operator: "Exists"
            effect: "NoSchedule"
        topologySpreadConstraints:
          - maxSkew: 1
            topologyKey: topology.kubernetes.io/zone
            whenUnsatisfiable: ScheduleAnyway
            labelSelector:
              matchLabels:
                strimzi.io/cluster: kafka-cluster
                strimzi.io/component-type: zookeeper
  kafkaExporter:
    topicRegex: ".*"
    groupRegex: ".*"
    template:
      pod:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                    - key: node-role.kubernetes.io/worker
                      operator: Exists
        tolerations:
          - key: "<XXX>"
            operator: "Exists"
            effect: "NoSchedule"

Can someone respond on this please ?