Release v0.20.0

🛑 Breaking changes 🛑

• operator: Migrate operator configuration from ConfigMap to environment variables (#1348)
  The default operator deployment no longer uses a ConfigMap-based configuration file.
  Configuration is now managed through environment variables.

  To continue using a ConfigMap: Mount your controller_manager_config.yaml ConfigMap and add
  the --config=/path/to/config.yaml flag to the operator deployment args.
  Environment variables will still take precedence over the config file.

  To migrate to environment variables: Use FEATURE_GATES for boolean flags (comma-separated,
  prefix with - to disable) and individual env vars for other settings.

  Example OLM Subscription override:

  spec:
    config:
      env:
      - name: FEATURE_GATES
        value: "prometheusOperator,observability.metrics.createServiceMonitors,-networkPolicies"
      - name: TLS_PROFILE
        value: "Intermediate"

  Available feature gates: openshift.route, openshift.servingCertsService, openshift.oauthProxy,
  httpEncryption, grpcEncryption, prometheusOperator, grafanaOperator, builtInCertManagement,
  observability.metrics.createServiceMonitors, observability.metrics.createPrometheusRules, networkPolicies

  Other env vars: DISTRIBUTION, TLS_PROFILE, OPENSHIFT_BASE_DOMAIN, DEFAULT_POD_SECURITY_CONTEXT (JSON),
  BUILT_IN_CERT_MANAGEMENT_CA_VALIDITY, BUILT_IN_CERT_MANAGEMENT_CA_REFRESH,
  BUILT_IN_CERT_MANAGEMENT_CERT_VALIDITY, BUILT_IN_CERT_MANAGEMENT_CERT_REFRESH

💡 Enhancements 💡

• tempostack, tempomonolithic: Add configurable defaultPodSecurityContext to operator config (#1240)
  The operator now supports a configurable defaultPodSecurityContext in the operator configuration.
  This allows setting default pod security context fields (fsGroup, runAsUser, etc.) that are
  merged into pod security contexts when specific fields are not set by the user.
  For Community distributions, fsGroup is set to 10001 to ensure volume permissions work correctly
  with certain CSI drivers (AWS EBS, DigitalOcean, etc.) that mount volumes with root:root ownership.
  For OpenShift, this is set to an empty object {} as SCCs manage security contexts automatically.

• tempostack, tempomonolithic: Add MCP (Model Context Protocol) server configuration to TempoStack and TempoMonolithic CRDs (#1319)
  The MCP server allows AI assistants to query tracing data from Tempo.
  Enable it by setting spec.template.queryFrontend.mcpServer.enabled: true for TempoStack
  or spec.query.mcpServer.enabled: true for TempoMonolithic.

• tempostack, tempomonolithic: Add support for extra labels on ServiceMonitor and PrometheusRule objects (#905)

• networking: Add network policy support for Tempo operands (#1246)
  Network policies are now enabled by default for all TempoStacks.

  Enabling/Disabling Network Policies

  Network policies are enabled by default. To disable them:

  apiVersion: tempo.grafana.com/v1alpha1
  kind: TempoStack
  metadata:
    name: example
  spec:
    networkPolicy:
      enabled: false  # Default is true

  Generated Network Policies

  The operator creates the following network policies for each component:

  Query-Frontend

  Egress (outbound connections):

  • Storage backend (S3/Azure/GCS) - Required for initialization
  • Queriers (ports 9095, 3200) - For query distribution
  • OTLP telemetry export (when configured)

  Ingress (inbound connections):

  • From cluster (port 9095, 3200) - External query access
  • From gateway (when enabled) - For proxied queries
  • From queriers (ports 9095, 3200) - Bidirectional worker communication

  Gateway (when enabled)

  Egress (outbound connections):

  • Distributor (ports 4317, 4318) - For ingesting traces via OTLP
  • Query-frontend (ports 9095, 3200) - For querying traces
  • Query-frontend (ports 16686, 16687) - For Jaeger Query UI and metrics (when JaegerQuery is enabled)
  • Kubernetes API server (port 6443) - For TokenReview/SubjectAccessReview (when OpenShift RBAC multi-tenancy is enabled)
  • OTLP telemetry export (when configured)

  Ingress (inbound connections):

  • From cluster (ports 8080, 8090) - External HTTP/gRPC access for trace ingestion and queries

  Ingester

  Egress (outbound connections):

  • Storage backend (S3/Azure/GCS) - For writing trace blocks
  • OTLP telemetry export (when configured)

  Ingress (inbound connections):

  • From distributor (ports 9095, 3200) - For receiving traces
  • From querier (ports 9095, 3200) - For querying recent traces

  Storage Backend Support

  • S3 (in-cluster): Port extracted from endpoint URL, defaults to 443 (HTTPS) or 80 (HTTP). Allows egress to any namespace/pod for services like MinIO.
  • Azure Storage: Port 443 (HTTPS). Allows egress to external internet (0.0.0.0/0) for Azure Storage endpoints.
  • Google Cloud Storage (GCS): Port 443 (HTTPS). Allows egress to external internet (0.0.0.0/0) for GCS and OAuth2 endpoints (storage.googleapis.com, oauth2.googleapis.com).

  Common Policies

  All components also have:

  • DNS egress (to kube-dns or openshift-dns)
  • Gossip ring communication (memberlist protocol)
  • Metrics ingress (port 3200) - For Prometheus/monitoring scraping from in-cluster monitoring namespaces

• tempostack, tempomonolithic: Show a warning if an instance without gateway is created on OpenShift (#1359)

• tempostack: Add t-shirt sizes for simplified TempoStack resource configuration (#1345)
  Introduces predefined deployment size profiles via the new spec.size field.
  Available sizes: 1x.demo, 1x.pico, 1x.extra-small, 1x.small, 1x.medium.
  Each size maps to pre-tested resource configurations based on performance testing.
  Size also sets a default replication factor (1 for demo, 2 for others).
  Resource values are based on ingestion rates: extra-small (~100GB/day), small (~500GB/day), medium (~2TB/day).

• operator: Update Kubernetes dependencies to v1.34.3 (#1324)

• tempostack, tempomonolithic: Update Tempo to 2.10.0 (#1375)
  Update Tempo to 2.10.0.
  Changelog: https://github.com/grafana/tempo/releases/tag/v2.10.0

🧰 Bug fixes 🧰

• tempostack: NetworkPolicies now dynamically discover Kubernetes API server endpoints and ports instead of hardcoding port 6443 (#1295)
  This fix resolves connectivity issues on managed Kubernetes services (e.g., Amazon EKS, Google GKE)
  that expose the Kubernetes API server on non-standard ports like 443 instead of the default 6443.

  What Changed

  The operator now automatically discovers the actual Kubernetes API server endpoints by querying the
  kubernetes EndpointSlice in the default namespace at reconcile time. This provides:

  • Dynamic port detection: Works with port 443 (EKS, some GKE configs), 6443 (standard K8s), or custom ports
  • Specific IP restrictions: When available, creates /32 CIDR rules for each control plane node IP
  • Fallback: Falls back to 0.0.0.0/0 with discovered port if EndpointSlice lookup fails

  Example

  On an EKS cluster with 3 control plane nodes, the operator now generates:

  spec:
    egress:
    - ports:
      - protocol: TCP
        port: 443  # Discovered from EndpointSlice
      to:
      - ipBlock:
          cidr: 100.105.216.2/32
      - ipBlock:
          cidr: 100.109.70.105/32
      - ipBlock:
          cidr: 100.114.146.103/32

  No configuration changes are required. The operator automatically adapts to cluster's API server configuration.

• tempomonolithic: Use the gateway serving cert for ingestion TLS if the gateway is enabled (#1372)

Components

• Tempo: v2.10.0

Support

This release supports Kubernetes 1.25 to 1.34.

Release v0.19.0

💡 Enhancements 💡

• tempostack, tempomonolithic: Update Tempo to 2.9.0 (#1308)

🧰 Bug fixes 🧰

• tempomonolithic: Scrape tempo metrics for monolithic. (#1275)
• tempostack: Restart pods when certificates are re-generated. (#1301)

Components

• Tempo: v2.9.0

Support

This release supports Kubernetes 1.25 to 1.32.

Release v0.18.0

💡 Enhancements 💡

• operator: Add feature gate to enable reconcile of Operator's default network policies. (#1248)
• tempostack, tempomonolithic: Update Tempo to 2.8.2 (#1276)
  Update Tempo to 2.8.2.
  Changelog: https://github.com/grafana/tempo/releases/tag/v2.8.2

Components

• Tempo: v2.8.2

Support

This release supports Kubernetes 1.25 to 1.32.

Release v0.17.1

🧰 Bug fixes 🧰

• github action: Fix release workflow (#1243)
  Fix the image tag of the must-gather image.

Components

• Tempo: v2.8.1

Support

This release supports Kubernetes 1.25 to 1.32.

Release v0.17.0

💡 Enhancements 💡

• tempomonolithic: Add attribute for configure tempo-query resources request and limits (#1227)

• tempomonolithic: Add storageClassName and podSecurityContext configuration options (#1231)

• tempostack, tempomonolithic: Update Tempo to 2.8.1 (#1238)
  Update Tempo to 2.8.1.
  Changelog: https://github.com/grafana/tempo/releases/tag/v2.8.0 and https://github.com/grafana/tempo/releases/tag/v2.8.1

• tempostack, tempomonolithic: Use SelfSubjectAccessReview instead of SubjectAccessReview when OpenShift multi-tenancy mode is enabled (#1241)

🧰 Bug fixes 🧰

• tempostack: Remove deprecated storage.trace.cache setting (#1136)

Components

• Tempo: v2.8.1

Support

This release supports Kubernetes 1.25 to 1.32.

Release v0.16.0

🛑 Breaking changes 🛑

• tempostack, tempomonolithic: Ensure the operator does not grant additional permissions when enabling OpenShift tenancy mode (resolves CVE-2025-2786) (#1145)
  Ensure the permissions the operator is granting to the Tempo Service Account
  do not exceed the permissions of the user creating (or modifying) the Tempo instance
  when enabling OpenShift tenancy mode.

  To enable the OpenShift tenancy mode, the user must have permissions to create TokenReview and SubjectAccessReview.

  This breaking change does not affect existing Tempo instances in the cluster.
  However, the required permissions are now mandatory when creating or modifying a TempoStack or TempoMonolithic CR.

💡 Enhancements 💡

• tempostack, tempomonolithic: Add short live token authentication for Azure Blob Storage (#1206)
  For use short live token on Azure, the secret should contain the following configuration:

  data:
  container:         # Azure blob storage container name
  account_name:      # Azure blob storage account name
  client_id:         # Azure managed identity clientID
  tenant_id:         # Azure tenant ID in which the managed identity lives.
  audience:          # (optional) Audience of the token, default to api://AzureADTokenExchange
  

• tempostack, tempomonolithic: Support for AWS STS via cloudcredential operator (#1159)

• tempostack, tempomonolithic: Add support for GCS Shot Live Token authentication. (#1141)
  Now storage secret for GCS can contain

  data:
    bucketname:         # Bucket name
    iam_sa:             # a name for your the Google IAM service account
    iam_sa_project_id:  # The project ID for your IAM service account.
  

• tempostack, tempomonolithic: Set GOMEMLIMIT to 80% of memory limit, if any (#1196)
  This golang variable indicate to GoLang GC to be more aggressive when it is reaching out the
  memory limits. This is a soft limit, so still can produce OOM, but reduces the possibility.

• operator: Kubernetes 1.32 enablement (#1157)

• tempomonolithic: Watch storage secrets for tempo monolithic (#1181)

🧰 Bug fixes 🧰

• tempostack, tempomonolithic: Add parameter to set audience in ID token for GCP Workload Identity Federation (#1209)
  Now that GCS token allow to set the audience, the secret configuration required channged, now it will require
  the following:

  data:
    bucketname:    # GCS Bucket  name
    audience:      # (Optional) default to openshift
    key.json:      # Credential file generated using gclient
  

  File key.json can be created using :

  gcloud iam workload-identity-pools create-cred-config \
    "projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>" \
    --service-account="<SERVICE_ACCOUNT_EMAIL>" \
    --credential-source-file=/var/run/secrets/storage/serviceaccount/token \
    --credential-source-type=text \
    --output-file="/tmp/key.json"
  

  credential-source-file= Should be pointing to /var/run/secrets/storage/serviceaccount/token which is the locationn
  operator mounts the projected volume.

• tempostack, tempomonolithic: Add namespace suffix to ClusterRole and ClusterRoleBinding of gateway (#1146)
  This resolves a naming conflict of the ClusterRole and ClusterRoleBinding when two TempoStack/TempoMonolithic instances with the same name, but in different namespaces are created.
  Only relevant when using multi-tenancy with OpenShift mode.

• tempostack, tempomonolithic: Fix pruning of cluster-scoped resources (#1168)
  Previously, when a non-multitenant TempoStack instance was created using the same name as an existing multitenant TempoStack instance, the operator erroneously deleted the Gateway ClusterRole and ClusterRoleBinding associated with the multitenant instance.

  With this change, cluster-scoped resources get an additional label app.kubernetes.io/namespace to signify the namespace of the TempoStack owning this cluster-scoped resource.

• tempostack, tempomonolithic: Cleanup gateway cluster roles and bindings after deleting tempo instance (#1190)
  Now the operator uses finalizer to clean up the cluster roles and bindings after deleting the tempo instance.

• tempostack, tempomonolithic: Allow OpenShift cluster admins to see all attributes when RBAC is enabled. (#1185)
  This change removes --opa.admin-groups=system:cluster-admins,cluster-admin,dedicated-admin
  from the OpenShift OPA configuration. This configures the OPA to always return
  all user's accessible namespaces required by the RBAC feature.

• tempostack, tempomonolithic: Don't set --opa.matcher=kubernetes_namespace_name when query RBAC is disabled (#1176)

• tempostack: Fix unimplemented per tenant retention and fix per tenant overrides after tempo 2.3 (#1134)
  In tempo 2.3 https://github.com/grafana/tempo/blob/main/CHANGELOG.md#v230--2023-10-30 they changes the overrides config
  which was not properly implemented in the operator.

  This patch also adds support for per tenant retention which was not implemented.

• tempostack, tempomonolithic: Assign a percentage of the resources to oauth-proxy if resources are not specified, fixed the name (#1107)

• tempostack: Limit granted permissions of the Tempo Service Account when enabling the Jaeger UI Monitor tab on OpenShift (resolves CVE-2025-2842) (#1144)
  Previously, the operator assigned the cluster-monitoring-view ClusterRole to the Tempo Service Account
  when the Prometheus endpoint of the Jaeger UI Monitor tab is set to the Thanos Querier on OpenShift.

  With this change, the operator limits the granted permissions to only view metrics of the namespace of the Tempo instance.
  Additionally, the recommended port of the Thanos Querier service changed from 9091 to 9092 (tenancy-aware port):
  .spec.template.queryFrontend.jaegerQuery.monitorTab.prometheusEndpoint: https://thanos-querier.openshift-monitoring.svc.cluster.local:9092.

  All existing installations, which have the Thanos Querier configured at port 9091, will be upgraded automatically to use port 9092.

• tempostack, tempomonolithic: Update Tempo to 2.7.2 (#1149)

Components

• Tempo: v2.7.2

Support

This release supports Kubernetes 1.25 to 1.32.

Release v0.15.3

💡 Enhancements 💡

• tempomonolithic: Add support for query RBAC (#1131)
  This feature allows users to apply query RBAC in the multitenancy mode.
  The RBAC allows filtering span/resource/scope attributes and events based on the namespaces which a user querying the data can access.
  For instance, a user can only see attributes from namespaces it can access.

  spec:
    query:
      rbac:
        enabled: true

Components

• Tempo: v2.7.1

Release v0.15.2

Components

• Tempo: v2.7.1

Release v0.15.1

Components

• Tempo: v2.7.1

Release v0.15.0

🛑 Breaking changes 🛑

• tempostack, tempomonolithic: Update Tempo to 2.7.0 (#1110)
  Update Tempo to 2.7.0 https://github.com/grafana/tempo/releases/tag/v2.7.0
  The Tempo instrumentation changed from Jaeger to OpenTelemetry with OTLP/http exporter.

  The spec.observability.tracing.jaeger_agent_endpoint is deprecated in favor of spec.observability.tracing.otlp_http_endpoint.

  spec:
    observability:
      tracing:
        jaeger_agent_endpoint: # Deprecated!
        sampling_fraction: "1"
        otlp_http_endpoint: http://localhost:4320

💡 Enhancements 💡

• tempostack: Add support for query RBAC when Gateway/multitenancy is used. (#1100)
  This feature allows users to apply query RBAC in the multitenancy mode.
  The RBAC allows filtering span/resource/scope attributes and events based on the namespaces which a user querying the data can access.
  For instance, a user can only see attributes from namespaces it can access.

  spec:
    template:
      gateway:
        enabled: true
        rbac:
          enabled: true

• operator: Remove kube-rbac-proxy (#1094)
  The image won't be available and won't be mantained, switched to use WithAuthenticationAndAuthorization

🧰 Bug fixes 🧰

• tempostack: Include insecure option and tls options when STS S3 token is enabled (#1109)
• tempostack, tempomonolithic: Assign a percentage of the resources to oauth-proxy if resources are not specified (#1107)

Components

• Tempo: v2.7.0