-
Notifications
You must be signed in to change notification settings - Fork 329
rke2-traefik - opentelemetry enablement error "Unable to create OpenTelemetry meter provider" #9990
Description
Environmental Info:
RKE2 Version:
Tested on both these versions
v1.33.7+rke2r3
rke2 version v1.34.5+rke2r1 (105ddbd880270e1edcf8ea26a73e1f9be922ec83)
rke2-traefik Version:
Node(s) CPU architecture, OS, and Version:
Linux cilium-test-0.homelab.infra 6.8.0-100-generic #100-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 13 16:40:06 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 24.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.4 LTS (Noble Numbat)"
VERSION_CODENAME=nobleCluster Configuration:
1 cp/etcd
2 workers
Describe the bug:
When running rke2-traefik chart version rke2-traefik-39.0.002 and enabling opentelemetry collector feature in the helm chart values.yaml. The opentelemetry sdk fails to build resource due to unknown user detection error.
root@cilium-test-0:~# kubectl logs ds/rke2-traefik -n traefik | grep -i error
Found 2 pods, using pod/rke2-traefik-zglc8
{"level":"error","metricsProviderName":"openTelemetry","error":"building resource: error detecting resource: user: unknown userid 65532","time":"2026-03-19T00:25:29Z","message":"Unable to create OpenTelemetry meter provider"}
{"level":"error","error":"building resource: error detecting resource: user: unknown userid 65532","time":"2026-03-19T00:25:29Z","message":"Unable to create OpenTelemetry meter provider"}
{"level":"warn","error":"building resource: error detecting resource: user: unknown userid 65532","time":"2026-03-19T00:25:29Z","message":"Unable to create tracer"}Steps To Reproduce:
- Installed RKE2:
- Kubectl apply helmchartconfig
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-traefik
namespace: kube-system
spec:
valuesContent: |-
env:
- name: USER
value: "nobody"
- name: POD_UID
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.uid
- name: OTEL_RESOURCE_ATTRIBUTES
value: "k8s.pod.name=$(POD_NAME),k8s.pod.uid=$(POD_UID),k8s.namespace.name=$(POD_NAMESPACE)"
global:
checkNewVersion: false
hostNetwork: true
logs:
access:
enabled: true
format: json
general:
format: json
level: INFO
noColor: true
metrics:
otlp:
enabled: true
http:
enabled: true
endpoint: http://test-cluster-telemetry-ingest.dynatrace.svc.cluster.local:4318/v1/metrics
tls: null
namespaceOverride: traefik
nodeSelector:
kubernetes.io/os: linux
ports:
traefik:
containerPort: 8192
hostPort: 8192
port: 8192
web:
containerPort: 8193
expose:
default: false
hostPort: 8193
port: 8193
websecure:
containerPort: 8194
expose:
default: true
hostPort: 8194
http:
tls:
options: default@file
port: 8194
providers:
kubernetesIngress:
enabled: true
ingressClass: traefik-native
publishedService:
enabled: false
kubernetesIngressNginx:
controllerClass: traefik.io/ingress-controller
enabled: true
ingressClass: traefik
tracing:
otlp:
enabled: true
http:
enabled: true
endpoint: "http://test-cluster-telemetry-ingest.dynatrace.svc.cluster.local:4318/v1/metrics"
tls: nullExpected behavior:
Enabling opentelemetry in rke2-traefik should work out of the box with no workarounds.
Actual behavior:
Opentelemetry resources aren't generated and traces are not sent.
Additional context / logs:
Upstream traefik had similar issue that was fixed. ref- traefik/traefik#11992.
This seems to be specific the hardened re-package of the treafik binary in a scratch container not having /etc/passwd.
If I mount /etc/passwd from the host as read only and null/remove the podSecurityContext functionality; opentelemetry works as expected.