Metrics Push interval is 15s instead of default 60s

[nabbdl]

Describe the bug

We are using the Java opentelemetry java instrumentation injected by opentelemetry operator on kubernetes. The metrics are pushed every 15s and we would like to lower this interval. According to the documentation the default interval is set to 60s. We tried to change the OTEL_METRIC_EXPORTER_INTERVAL variable without success. We then enable the debug log (OTLEL_JAVAAGENT_DEBUG), and we can see that the INTERVAL (160000000000) variable is taken into account but it has no effects :-(

OBSERVABLE_GAUGE=default, GAUGE=default}, memoryMode=REUSABLE_DATA}, intervalNanos=160000000000}]

We are currently running the latest version of the agent and we tried older version without success.

Steps to reproduce

Install Opentelemetry operator in Kubernetes
Created instrumentation configuration:
spec:
exporter:
endpoint: http://xxxxxxxxxxxxxxxxxxxxx:4318
propagators:
- tracecontext
- baggage
sampler:
type: parentbased_always_on
java:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:latest
env:
- name: OTEL_SERVICE_NAME
valueFrom:
fieldRef:
fieldPath: "metadata.labels['app.kubernetes.io/instance']"
- name: OTEL_TRACES_EXPORTER
value: otlp
- name: OTEL_METRICS_EXPORTER
value: otlp
- name: OTEL_LOGS_EXPORTER
value: none
- name: OTEL_EXPORTER_OTLP_PROTOCOL
value: http/protobuf
- name: OTEL_INSTRUMENTATION_COMMON_MDC_RESOURCE_ATTRIBUTES
value: service.namespace,service.name,service.instance.id,service.version,deployment.environment

Added the inject annotation on a kubernetes deployment
Start a java app
Observ the metrics interval on grafana

Expected behavior

Be able to change the metric push interval.

Actual behavior

Metric effective push interval is 15s

Javaagent or library instrumentation version

2.12.0

Environment

JDK:
OS:

Additional context

Tried with multiple Java apps and getting the same issue.

[laurit]

We tried to change the OTEL_METRIC_EXPORTER_INTERVAL variable without success.

See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader The environment variable is OTEL_METRIC_EXPORT_INTERVAL not OTEL_METRIC_EXPORTER_INTERVAL. Was this a typo in the issue description?

we can see that the INTERVAL (160000000000) variable is taken into account but it has no effects :-(

What did you expect to happen when you set the export interval? How did you verify that it has no effect?

[nabbdl]

Sorry for that. You're right it's a typo. I tested using OTEL_METRIC_EXPORT_INTERVAL.
I'm expecting to get metrics in the configured interval (160s in the example). Instead the metrics are pushed every 15s. Checked on grafana

[laurit]

I'm expecting to get metrics in the configured interval (160s in the example). Instead the metrics are pushed every 15s. Checked on grafana

The dots on the grafana chart could mean anything. Try sending to otel collector that is configured with debug logging and from the collector output determine whether the metrics are sent with the expected interval.

[nabbdl]

You were right, the dots on the charts do not represent periodidicity. By activating debugging on the collector I was able to see that the interval is ok. Sorry for the inconvenience and thank you for your help