Skip to content

kubectl apply: Deleting one instance of a duplicate environment variable removes all occurrences – breaking change affecting Helm, Helmfile, ArgoCD, and automation toolchains #1766

@JuryA

Description

@JuryA

What happened:

When an environment variable (e.g., NODE_ENV) appears multiple times in the env array in a Kubernetes manifest, and one duplicate is removed, running kubectl apply will remove all instances of that variable from the live resource. Only after a subsequent apply (with the deduplicated manifest) will the variable be added again.

This is a breaking change compared to prior behavior, where the first instance was preserved. In modern deployment pipelines, manifests are often generated automatically, and such duplicities can easily occur.

Important

This change fundamentally alters the behavior of kubectl apply regarding duplicate environment variables—a core functionality used daily by thousands of users for over a decade.

Why this is a critical problem:

  • Automated toolchains (Helm, Helmfile, ArgoCD, Flux, GitHub Actions, CI/CD, GitOps, etc.) often generate or process manifests where accidental duplicates are possible and hard to eliminate completely.
  • Workaround ("use server-side apply") is not acceptable – many toolchains, including Helm and ArgoCD, rely on the default client-side apply and cannot easily switch to server-side apply.
  • This behavior can cause outages or severe misconfiguration in production, as critical environment variables may be silently removed from running workloads.
  • Detection is difficult: The issue is not obvious and may lead to silent failures, security issues, or outages if environment variables disappear unexpectedly.
  • This is not backwards compatible with previous kubectl/Kubernetes behavior and is a regression for many users.

How to reproduce it (minimized):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx
          env:
            - name: NODE_ENV
              value: "qa"
            - name: NODE_ENV
              value: "qa"
  1. kubectl apply -f deployment.yaml # creates deployment with duplicated env
  2. Remove one of the NODE_ENV envs from the manifest.
  3. kubectl apply -f deployment.yaml # both NODE_ENV are removed from the live object
  4. kubectl apply -f deployment.yaml # only now is NODE_ENV re-added

Expected behavior:

  • Removing one duplicate should only remove that instance; at least one instance of the env var should remain.
  • Ideally, apply should error on duplicate keys in lists, or at minimum, preserve the first occurrence, and never silently remove all.

Environment:

  • Kubernetes Client Version: v1.30.5
  • Kubernetes Server Version: v1.30-v1.33
  • OS: Linux

Why the current workaround is insufficient

  • Not all toolchains support server-side apply.
  • Helm, Helmfile, ArgoCD and many others cannot simply be reconfigured.
  • Manual linting or deduplication is not always feasible in complex CI/CD environments.

Request

  • Please consider this a critical regression and re-evaluate the merge/diff algorithm.
  • At minimum, kubectl apply should fail with a clear error on duplicated items in lists (such as env).
  • Never silently remove all instances of a variable if just one was removed from the input manifest.

Related


This bug can cause production outages and is not a corner case. Please re-open or escalate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions