Skip to content

spec.podAnnotations entries can not be deleted in reconcile #2795

@THUzxj

Description

@THUzxj

Component(s)

collector

What happened?

Description

Hi, I tested opentelemetry operator with testing tool acto, which automatically generates otelcol instance definition yaml files and performs end-to-end testing. Configurations of acto to test opentelemetry operator are here.

I found that spec.podAnnotations entries can not be deleted when the user applies new definitions to reconcile.

Steps to Reproduce

  1. create a kind cluster
  2. install cert-manager and opentelemetry-operator with kubectl apply -f
  3. create and change the definition of opentelemetry collector one by one as follows (mutation 1 and 2 are the most important):
# mutation 0
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: test-cluster
spec:
  config: "receivers:\n  otlp:\n    protocols:\n      grpc:\n      http:\nprocessors:\n\
    \  memory_limiter:\n    check_interval: 1s\n    limit_percentage: 75\n    spike_limit_percentage:\
    \ 15\n  batch:\n    send_batch_size: 10000\n    timeout: 10s\n\nexporters:\n \
    \ debug:\n\nservice:\n  pipelines:\n    traces:\n      receivers: [otlp]\n   \
    \   processors: []\n      exporters: [debug]"
# mutation 1
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: test-cluster
spec:
  config: "receivers:\n  otlp:\n    protocols:\n      grpc:\n      http:\nprocessors:\n\
    \  memory_limiter:\n    check_interval: 1s\n    limit_percentage: 75\n    spike_limit_percentage:\
    \ 15\n  batch:\n    send_batch_size: 10000\n    timeout: 10s\n\nexporters:\n \
    \ debug:\n\nservice:\n  pipelines:\n    traces:\n      receivers: [otlp]\n   \
    \   processors: []\n      exporters: [debug]"
  podAnnotations:
    actokey: actokey
# mutation 2
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: test-cluster
spec:
  config: "receivers:\n  otlp:\n    protocols:\n      grpc:\n      http:\nprocessors:\n\
    \  memory_limiter:\n    check_interval: 1s\n    limit_percentage: 75\n    spike_limit_percentage:\
    \ 15\n  batch:\n    send_batch_size: 10000\n    timeout: 10s\n\nexporters:\n \
    \ debug:\n\nservice:\n  pipelines:\n    traces:\n      receivers: [otlp]\n   \
    \   processors: []\n      exporters: [debug]"
  podAnnotations: {}
# mutation 3
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: test-cluster
spec:
  config: "receivers:\n  otlp:\n    protocols:\n      grpc:\n      http:\nprocessors:\n\
    \  memory_limiter:\n    check_interval: 1s\n    limit_percentage: 75\n    spike_limit_percentage:\
    \ 15\n  batch:\n    send_batch_size: 10000\n    timeout: 10s\n\nexporters:\n \
    \ debug:\n\nservice:\n  pipelines:\n    traces:\n      receivers: [otlp]\n   \
    \   processors: []\n      exporters: [debug]"
  podAnnotations:
    actokey2: actokey2

Expected Result

In the state mutation 2 and mutation 3, the spec.podAnnotations entry actokey: actokey in user specification is been deleted, and the annotations of pod and deployment should be changed as what spec.podAnnotations defines.

Actual Result

When I describe the pod and deployment test-cluster-collector after applying mutation 2 and mutation 3, the actokey: actokey still exists. In mutation 3 I can see both actokey: actokey and actokey2: actokey2 in the annotations.

Kubernetes Version

1.28.0

Operator version

0.95.0

Collector version

0.95.0

Environment information

Environment

OS: Ubuntu 22.04

Log output

No response

Additional context

When logging the variables related to the annotations in the operator (like desiredObjects in this line(

func reconcileDesiredObjects(ctx context.Context, kubeClient client.Client, logger logr.Logger, owner metav1.Object, scheme *runtime.Scheme, desiredObjects []client.Object, ownedObjects map[types.UID]client.Object) error {
)), we can find that the data are changed as expected. Thus maybe the problem is related to the k8s controller-runtime API in this line(
result, createOrUpdateErr := ctrl.CreateOrUpdate(ctx, kubeClient, existing, mutateFn)
).

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions