Deploy Redpanda for Production in Kubernetes

This topic describes how to configure and deploy one or more Redpanda clusters and Redpanda Console in Kubernetes.

Prerequisites

Make sure that your Kubernetes cluster meets the requirements.

You must already have a ConfigMap that stores your io-config.yaml file. See Generate optimal I/O configuration settings.

Deploy a Redpanda cluster

To deploy Redpanda and Redpanda Console, you can use the following tools:

Helm and the Redpanda Operator: The Redpanda Operator extends Kubernetes with custom resource definitions (CRDs), allowing you to define Redpanda clusters as native Kubernetes resources. The resource that the Redpanda Operator uses to represent a Redpanda cluster is the Redpanda resource.
Helm: Helm is a package manager for Kubernetes, which simplifies the process of defining, installing, and upgrading Kubernetes applications. Helm uses charts, a collection of files that describe a related set of Kubernetes resources, to deploy applications in a Kubernetes cluster.

Tip	For more details about the differences between these two methods, see ./k-deployment-overview.adoc.

Helm + Operator

Note	The Redpanda Operator is namespace scoped. You must install the Redpanda Operator in the same namespace as your Redpanda resource (Redpanda cluster).

Make sure that you have permission to install custom resource definitions (CRDs):
```
kubectl auth can-i create CustomResourceDefinition --all-namespaces
```
You should see yes in the output.

You need these cluster-level permissions to install glossterm:cert-manager[^] and Redpanda Operator CRDs in the next steps.

Install cert-manager:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --set crds.enabled=true \
  --namespace cert-manager  \
  --create-namespace

The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates.

Install the Redpanda Operator CRDs:

deploy:partial$kubernetes/install-crds.adoc
Create a YAML file to override the default version of the Redpanda Operator.
redpanda-operator-values.yaml
```
image:
  tag: {latest-operator-version}
```

Deploy the Redpanda Operator.

helm repo add redpanda https://charts.redpanda.com
helm upgrade --install redpanda-controller redpanda/operator \
  --namespace <namespace> \
  --create-namespace \
  --values redpanda-operator-values.yaml

Ensure that the Deployment is successfully rolled out:

kubectl --namespace <namespace> rollout status --watch deployment/redpanda-controller-operator

deployment "redpanda-controller-operator" successfully rolled out

Install a Redpanda custom resource to deploy a Redpanda cluster and Redpanda Console.
redpanda-cluster.yaml
```
apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef:
    chartVersion: {latest-redpanda-helm-chart-version}
  clusterSpec:
    #useFlux: true
    #enterprise:
      #licenseSecretRef:
        #name: <secret-name>
        #key: <secret-key>
    statefulset:
      extraVolumes: |-
        - name: redpanda-io-config
          configMap:
            name: redpanda-io-config
      extraVolumeMounts: |-
        - name: redpanda-io-config
          mountPath: /etc/redpanda-io-config
      additionalRedpandaCmdFlags:
        - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"
```
- metadata.name: Name to assign the Redpanda cluster. This name is also assigned to the Helm release.
- spec.chartRef: Information about the Helm chart that will be used to deploy Redpanda.
- spec.chartRef.chartVersion: This field specifies the exact version of the Redpanda Helm chart to use for deployment. By setting this value, you pin the chart to a specific version, which prevents automatic updates that might introduce breaking changes or new features that have not been tested in your environment.
- spec.clusterSpec: This is where you can override default values in the Redpanda Helm chart. Here, you mount the I/O configuration file to the Pods that run Redpanda. For other configuration details, see Production considerations.
- spec.clusterSpec.useFlux: By default, the Redpanda Operator uses Flux controllers to deploy and manage the Redpanda resource. Set this to false to disable Flux and instead use the Redpanda Operator controllers.
- spec.clusterSpec.enterprise: If you want to use enterprise features in Redpanda, uncomment this section and add the details of a Secret that stores your Enterprise Edition license key. For details, see get-started:licenses.adoc.
- spec.clusterSpec.statefulset: Here, you mount the I/O configuration file to the Pods that run Redpanda. For other configuration details, see Production considerations.
Apply the Redpanda resource:
```
kubectl apply -f redpanda-cluster.yaml --namespace <namespace>
```
Note
The Redpanda resource must be deployed in the same namespace as the Redpanda Operator. Each new deployment of Redpanda requires a separate namespace.
Wait for the Redpanda Operator to deploy Redpanda using the Helm chart:
```
kubectl get redpanda --namespace <namespace> --watch
```
```
NAME       READY   STATUS
redpanda   True    Redpanda reconciliation succeeded
```
This step may take a few minutes. You can watch for new Pods to make sure that the deployment is progressing:
```
kubectl get pod --namespace <namespace>
```
If it’s taking too long, see Troubleshooting.

Verify that each Redpanda broker is scheduled on only one Kubernetes node:

kubectl get pod --namespace <namespace>  \
  -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
  app.kubernetes.io/component=redpanda-statefulset

Expected output:

example-worker3   redpanda-0
example-worker2   redpanda-1
example-worker    redpanda-2

Helm

Install cert-manager using Helm:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --set crds.enabled=true \
  --namespace cert-manager  \
  --create-namespace

The Redpanda Helm chart enables TLS by default and uses cert-manager to manage TLS certificates.

Override the default values to mount your I/O configuration file onto each Pod that runs Redpanda.

redpanda-values.yaml

statefulset:
  extraVolumes: |-
    - name: redpanda-io-config
      configMap:
        name: redpanda-io-config
  extraVolumeMounts: |-
    - name: redpanda-io-config
      mountPath: /etc/redpanda-io-config
  additionalRedpandaCmdFlags:
    - "--io-properties-file=/etc/redpanda-io-config/io-config.yaml"

Redpanda reads from this file at startup to optimize itself for the given I/O parameters.

If you want to use enterprise features in Redpanda, add the details of a Secret that stores your Enterprise Edition license key.

redpanda-values.yaml

enterprise:
  licenseSecretRef:
    name: <secret-name>
    key: <secret-key>

For details, see get-started:licensing/add-license-redpanda/kubernetes.adoc.

Install the Redpanda Helm chart to deploy a Redpanda cluster and Redpanda Console.
```
helm repo add redpanda https://charts.redpanda.com
helm install redpanda redpanda/redpanda \
  --version {latest-redpanda-helm-chart-version} \ (1)
  --namespace <namespace> \ (2)
  --create-namespace \
  --values redpanda-values.yaml
```
1. This flag specifies the exact version of the Redpanda Helm chart to use for deployment. By setting this value, you pin the chart to a specific version, which prevents automatic updates that might introduce breaking changes or new features that have not been tested in your environment.
2. Each deployment of the Redpanda Helm chart requires a separate namespace. Ensure you choose a unique namespace for each deployment.

Wait for the Redpanda cluster to be ready:

kubectl --namespace <namespace> rollout status statefulset redpanda --watch

When the Redpanda cluster is ready, the output should look similar to the following:

statefulset rolling update complete 3 pods at revision redpanda-8654f645b4...

Verify that each Redpanda broker is scheduled on only one Kubernetes node:

kubectl get pod --namespace <namespace> \
-o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name -l \
app.kubernetes.io/component=redpanda-statefulset

Expected output:

example-worker3   redpanda-0
example-worker2   redpanda-1
example-worker    redpanda-2

Deploy multiple Redpanda clusters

You can deploy more than one Redpanda cluster in the same Kubernetes cluster by using a different namespace and unique node ports.

Helm + Operator

Install another instance of the Redpanda Operator in a different namespace to your existing ones. This Redpanda Operator will manage Redpanda clusters only in its namespace.

helm repo add redpanda https://charts.redpanda.com
helm upgrade --install redpanda-controller redpanda/operator \
  --namespace <another-namespace> \
  --set image.tag={latest-operator-version} \
  --create-namespace

Apply a Redpanda resource in the same namespace as your new Redpanda Operator to deploy your new Redpanda cluster.

Note	Make sure to use unique node ports for the listeners in your Redpanda resource so that they don’t conflict with any existing node ports in your other Redpanda clusters. See External access.

redpanda-cluster-two.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda-two
spec:
  chartRef:
    chartVersion: {latest-redpanda-helm-chart-version}
  clusterSpec:
    listeners:
      kafka:
        external:
          default:
            advertisedPorts: [31093]
      admin:
        external:
          default:
            advertisedPorts: [31645]
      http:
        external:
          default:
            advertisedPorts: [30083]
      rpc:
        port: 33146
      schemaRegistry:
        external:
          default:
            advertisedPorts: [30084]

Helm

Install the Redpanda Helm chart in a different namespace to your existing Redpanda clusters.

Note	Make sure to use unique node ports for the listeners in your Redpanda resource so that they don’t conflict with any existing node ports in your other Redpanda clusters. See External access.

helm repo add redpanda https://charts.redpanda.com
helm install redpanda-two redpanda/redpanda \
  --version {latest-redpanda-helm-chart-version} \
  --namespace <anothernamespace> \
  --set listeners.kafka.external.default.advertisedPorts[0]=31093 \
  --set listeners.admin.external.default.advertisedPorts[0]=31645 \
  --set listeners.http.external.default.advertisedPorts[0]=30083 \
  --set listeners.rpc.port=33146 \
  --set listeners.schemaRegistry.external.default.advertisedPorts[0]=30084
  --create-namespace

Production considerations

This section provides advice for configuring the Redpanda Helm chart for production. For all available settings, see reference:k-redpanda-helm-spec.adoc. To learn how to customize the Redpanda Helm chart, see Customize the Helm Chart.

Version pinning

The Redpanda application version, which is what the Helm chart deploys, can change even in patch versions of the Helm chart. This means that updates to the chart may potentially roll out new versions of Redpanda with significant changes that could impact your systems.

To avoid unexpected changes to your deployments, pin the version of the Helm chart. Pinning refers to the practice of specifying an exact version to use during deployment, rather than using the latest or unspecified version. When you pin the Helm chart version, you maintain consistent, predictable environments, especially in production. Using a specific version helps to:

Ensure compatibility: Guarantee that the deployed application behaves as tested, regardless of new chart versions being released.
Avoid unexpected updates: Prevent automatic updates that may introduce changes incompatible with the current deployment or operational practices.

Helm + Operator

redpanda-cluster.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Redpanda
metadata:
  name: redpanda
spec:
  chartRef:
    chartVersion: {latest-redpanda-helm-chart-version}

Helm

helm repo add redpanda https://charts.redpanda.com
helm install redpanda redpanda/redpanda \
  --version {latest-redpanda-helm-chart-version} \
  --namespace <namespace> \
  --create-namespace

Review the release notes to understand any significant changes, bug fixes, or potential disruptions that could affect your existing deployment.

Name overrides

Deploying multiple instances of the same Helm chart in a Kubernetes cluster can lead to naming conflicts. Using nameOverride and fullnameOverride helps differentiate between them. If you have a production and staging environment for Redpanda, different names help to avoid confusion.

Use nameOverride to customize the labels app.kubernetes.io/component=<nameOverride>-statefulset and app.kubernetes.io/name=<nameOverride>.
Use fullnameOverride to customize the name of the StatefulSet and Services.

nameOverride: 'redpanda-production'
fullnameOverride: 'redpanda-instance-prod'

For all available settings, see the Helm specification.

Labels

Kubernetes labels help you to organize, query, and manage your resources. Use labels to categorize Kubernetes resources in different deployments by environment, purpose, or team.

commonLabels:
  env: 'production'

For all available settings, see the Helm specification.

Tolerations

Tolerations and taints allow Pods to be scheduled onto nodes where they otherwise wouldn’t. If you have nodes dedicated to Redpanda with a taint dedicated=redpanda:NoSchedule, the following toleration allows the Redpanda brokers to be scheduled on them.

tolerations:
- key: "dedicated"
  operator: "Equal"
  value: "redpanda"
  effect: "NoSchedule"

For all available settings, see the Helm specification.

Docker image

You can specify the image tag to deploy a known version of the Redpanda Docker image. By default, the image tag is set in Chart.appVersion. Avoid using the latest tag, which can lead to unexpected changes.

If you’re using a private repository, always ensure your nodes have the necessary credentials to pull the image.

image:
  repository: docker.redpanda.com/redpandadata/redpanda
  tag: "{latest-redpanda-tag}"
imagePullSecrets: []

For all available settings, see the Helm specification.

Number of Redpanda brokers

By default, the Redpanda Helm chart deploys a StatefulSet with three Pod replicas (Redpanda brokers). In a production cluster, deploy at least three Redpanda brokers to use as glossterm:seed server[]. A larger number of seed servers makes consensus more robust and minimizes the chance of unwanted clusters forming when brokers are restarted without any data.

statefulset:
  replicas: 3

Note	You must provision one dedicated worker node for each Redpanda broker that you plan to deploy in your Redpanda cluster. The Redpanda Helm chart configures `podAntiAffinity` rules to make sure that each Redpanda broker runs on its own worker node.

For all available settings, see the Helm specification.

See also:

TLS

By default, the Helm chart enables TLS (Transport Layer Security) for encrypted communication. Internal (default) and external (external) self-signed certificates are generated using cert-manager. See [TLS Certificates].

tls:
  enabled: true
  certs:
    # This key represents the name of the certificate.
    default:
      caEnabled: true
    # This key represents the name of the certificate.
    external:
      caEnabled: true

For all available settings, see the Helm specification.

See also: manage:kubernetes/security/kubernetes-tls.adoc

Authentication

If you want to authenticate clients connections to the Redpanda cluster, you can enable SASL authentication.

auth:
  sasl:
    enabled: true
    mechanism: "SCRAM-SHA-512"
    secretRef: "sasl-password-secret"
    users: []

For all available settings, see the Helm specification.

See also: manage:kubernetes/security/authentication/k-authentication.adoc

Resources

By default, the resources allocated to Redpanda are for a development environment. In a production cluster, the resources you allocate should be proportionate to your machine type. You should determine and set these values before deploying the cluster.

resources:
  cpu:
    cores: 4
  memory:
    enable_memory_locking: true
    container:
      max: 10Gi

For all available settings, see the Helm specification.

See also:

manage:kubernetes/k-manage-resources.adoc
deploy:deployment-option/self-hosted/kubernetes/k-requirements.adoc

Storage

By default, the Redpanda Helm chart creates PersistentVolumeClaims (PVCs) for each Redpanda broker in the StatefulSet. The PVC uses the default StorageClass in your cluster.

In production, it’s best to use local PersistentVolumes (PVs) that are backed by NVMe devices to store the Redpanda data directory. NVMe devices outperform traditional SSDs or HDDs. For cloud instance types that support NVMe disks, see ./cloud-instance-local-storage.adoc.

Redpanda Data recommends creating StorageClasses that use the local volume manager (LVM) CSI driver to automatically provision PVs. The LVM allows you to group physical storage devices into a logical volume group. Allocating logical volumes from a logical volume group provides greater flexibility in terms of storage expansion and management. The LVM supports features such as resizing, snapshots, and striping, which are not available with the other drivers such as the local volume static provisioner.

storage:
  persistentVolume:
    enabled: true
    size: 100Gi
    storageClass: csi-driver-lvm-striped-xfs

Tip	For an example of configuring local PersistentVolumes backed by NVMe disks, see one of the following guides: Azure Kubernetes Service (AKS) Elastic Kubernetes Service (EKS) Google Kubernetes Engine (GKE)

For all available settings, see the Helm specification.

See also:

manage:kubernetes/storage/k-volume-types.adoc
manage:kubernetes/storage/k-configure-storage.adoc

External access

By default, the Redpanda Helm chart deploys a NodePort Service for external access to the Redpanda listeners. You can configure individual node ports for each listener.

The NodePort Service provides the lowest latency of all the Kubernetes Services because it does not include any unnecessary routing or middleware. Client connections go to the Redpanda brokers in the most direct way possible, through the worker nodes.

By default, the fully qualified domain names (FQDNs) that brokers advertise are their internal addresses within the Kubernetes cluster, which are not reachable from outside the cluster. To make the cluster accessible from outside, each broker must advertise a domain that can be reached from outside the cluster.

external:
  enabled: true
  type: NodePort

For all available settings, see the Helm specification.

See also:

manage:kubernetes/networking/k-networking-and-connectivity.adoc
manage:kubernetes/networking/k-configure-listeners.adoc

ExternalDNS

You should use ExternalDNS to manage DNS records for your Pods' domains. ExternalDNS synchronizes exposed Kubernetes Services with various DNS providers, rendering Kubernetes resources accessible through DNS servers.

Benefits of ExternalDNS include:

Automation: ExternalDNS automatically configures public DNS records when you create, update, or delete Kubernetes Services or Ingresses. This eliminates the need for manual DNS configuration, which can be error-prone.
Compatibility: ExternalDNS is compatible with a wide range of DNS providers, including major cloud providers such as AWS, Google Cloud, and Azure, and DNS servers like CoreDNS and PowerDNS.
Integration with other tools: ExternalDNS can be used with other Kubernetes tools, such as ingress controllers or cert-manager for managing TLS certificates.

external:
  enabled: true
  type: LoadBalancer
  externalDns:
    enabled: true

For all available settings, see the Helm specification.

See also:

ExternalDNS with a NodePort Service
ExternalDNS with LoadBalancer Services

Logging

By default, the log-level is set to info. In production, use the info logging level to avoid overwhelming the storage. For debugging purposes, temporarily change the logging level to debug.

logging:
  level: "info"

For all available settings, see the Helm specification.

Monitoring

By default, monitoring is disabled. If you have the Prometheus Operator, enable monitoring to deploy a ServiceMonitor resource for Redpanda. Observability is essential in production environments.

monitoring:
  enabled: true

For all available settings, see the Helm specification.

See also: manage:kubernetes/monitor.adoc

StatefulSet update strategy

For smooth and uninterrupted updates, use the default RollingUpdate strategy. Additionally, set a budget to ensure a certain number of replicas stay available during the update.

statefulset:
  updateStrategy:
    type: "RollingUpdate"
  budget:
    maxUnavailable: 1

For all available settings, see the Helm specification.

Affinity rules

By default, the Redpanda Helm chart also uses podAntiAffinity rules to stop the Kubernetes scheduler from placing multiple Redpanda brokers on the same node. These rules offer two benefits:

To minimize the risk of data loss by ensuring that a node’s failure results in the loss of only one Redpanda broker.
To prevent resource contention between brokers by ensuring they are never co-located on the same node.

Affinities control Pod placement in the cluster based on various conditions. Set these according to your high availability and infrastructure needs. For example, this is a soft rule that tries to ensure the Kubernetes scheduler doesn’t place two Pods with the same app: redpanda label in the same zone. However, if it’s not possible, the scheduler can still place the Pods in the same zone.

statefulset:
  podAntiAffinity:
    topologyKey: kubernetes.io/hostname
    type: hard
    weight: 100
    custom:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: "app"
              operator: "In"
              values:
              - "redpanda"
          topologyKey: "kubernetes.io/zone"

For all available settings, see the Helm specification.

Graceful shutdown

By default, Pods are given 90 seconds to shut down gracefully. If your brokers require additional time for a graceful shutdown, modify the terminationGracePeriodSeconds.

statefulset:
  terminationGracePeriodSeconds: 100

For all available settings, see the Helm specification.

See also: upgrade:k-rolling-upgrade.adoc

Service account

Restricting permissions is a best practice. Create a dedicated ServiceAccount for each Pod. To assign roles to this ServiceAccount, see Role-based access control (RBAC).

serviceAccount:
  create: true
  name: "redpanda-service-account"

For all available settings, see the Helm specification.

Role-based access control (RBAC)

RBAC is a method for providing permissions to ServiceAccounts based on roles. Some features such as rack awareness require both a ServiceAccount and RBAC to access resources using the Kubernetes API.

rbac:
  enabled: true
  annotations: {}

Note	If you use the Redpanda Operator, you must also deploy the Redpanda Operator Helm chart with `rbac.createRPKBundleCRs` set to `true` to give it the required roles.

For all available settings, see the Helm specification.

See also: manage:kubernetes/k-rack-awareness.adoc

Find the latest versions of the Redpanda Helm charts

To list the latest version of the Redpanda Helm chart, use the helm search command:

helm search repo redpanda

To find the versions that are installed on your machine, run the following:

helm list --namespace <namespace>

deploy:partial$self-test.adoc

deploy:partial$kubernetes/default-components.adoc

deploy:partial$kubernetes/guides/uninstall.adoc

deploy:partial$kubernetes/guides/troubleshoot.adoc

Next steps

See the Manage Kubernetes topics to learn how to customize the chart to meet your needs.

shared:partial$suggested-reading.adoc

./k-high-availability.adoc
Redpanda Helm Specification
Redpanda CRD Reference

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy Redpanda for Production in Kubernetes

Prerequisites

Deploy a Redpanda cluster

Deploy multiple Redpanda clusters

Production considerations

Version pinning

Name overrides

Labels

Tolerations

Docker image

Number of Redpanda brokers

TLS

Authentication

Resources

Storage

External access

ExternalDNS

Logging

Monitoring

StatefulSet update strategy

Affinity rules

Graceful shutdown

Service account

Role-based access control (RBAC)

Find the latest versions of the Redpanda Helm charts

Next steps

FilesExpand file tree

k-production-deployment.adoc

Latest commit

History

k-production-deployment.adoc

File metadata and controls

Deploy Redpanda for Production in Kubernetes

Prerequisites

Deploy a Redpanda cluster

Deploy multiple Redpanda clusters

Production considerations

Version pinning

Name overrides

Labels

Tolerations

Docker image

Number of Redpanda brokers

TLS

Authentication

Resources

Storage

External access

ExternalDNS

Logging

Monitoring

StatefulSet update strategy

Affinity rules

Graceful shutdown

Service account

Role-based access control (RBAC)

Find the latest versions of the Redpanda Helm charts

Next steps