Skip to content

document edot java logging #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
theletterf merged 9 commits into from
Jun 27, 2025
Merged

document edot java logging #315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4963161
document declarative config as not supported
SylvainJuge Jun 24, 2025
79aa583
reformat table
SylvainJuge Jun 24, 2025
86933f0
wip
SylvainJuge Jun 25, 2025
108ea7d
slightly better wording
SylvainJuge Jun 25, 2025
82a0005
add 'application' config option
SylvainJuge Jun 25, 2025
076ea61
Apply suggestions from code review
SylvainJuge Jun 27, 2025
8342696
move agent logging to configuration
SylvainJuge Jun 27, 2025
f534dd6
Merge branch 'main' of github.com:elastic/opentelemetry into doc_java…
SylvainJuge Jun 27, 2025
b9f89ab
fix link
SylvainJuge Jun 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 27 additions & 6 deletions docs/reference/edot-sdks/java/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,13 @@ EDOT Java uses different defaults than the OpenTelemetry Java instrumentation fo
The EDOT Java instrumentation agent also provides configuration options for each of the [supported features](./features.md).
This table only contains minimal configuration, see each respective feature for exhaustive configuration options documentation.

| Option | Default | Feature |
|--------------------------------------------------------|---------|------------------------------------------------------------------------------------------------------|
| `OTEL_INFERRED_SPANS_ENABLED` | `false` | [Inferred spans](./features.md#inferred-spans) |
| `OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION` | `5ms` | [Span stacktrace](./features.md#span-stacktrace) |
| `ELASTIC_OTEL_UNIVERSAL_PROFILING_INTEGRATION_ENABLED` | `auto` | [Elastic Universal profiling integration](./features.md#elastic-universal-profiling-integration) |
| `OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED` | `true` | [OpenAI client instrumentation](./supported-technologies.md#openai-client-instrumentation) |
| Option | Default | Feature |
|--------------------------------------------------------|---------|--------------------------------------------------------------------------------------------------|
| `OTEL_INFERRED_SPANS_ENABLED` | `false` | [Inferred spans](./features.md#inferred-spans) |
| `OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION` | `5ms` | [Span stacktrace](./features.md#span-stacktrace) |
| `ELASTIC_OTEL_UNIVERSAL_PROFILING_INTEGRATION_ENABLED` | `auto` | [Elastic Universal profiling integration](./features.md#elastic-universal-profiling-integration) |
| `OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED` | `true` | [OpenAI client instrumentation](./supported-technologies.md#openai-client-instrumentation) |
| `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` | `INFO` | [Agent logging](./configuration.md#agent-logging) |

## OpenAI Client settings

Expand All @@ -97,6 +98,10 @@ Configuration options are applied with the following priorities:
- [environment variables](#system-properties) take precedence over [system properties](#system-properties) and [properties configuration file](#properties-configuration-file).
- [system properties](#system-properties) take precedence on [properties configuration file](#properties-configuration-file).

:::{important}
[Declarative configuration](https://opentelemetry.io/docs/specs/otel/configuration/#declarative-configuration) is not supported.
:::

### Environment variables

Environment variables provide a cross-platform way to configure EDOT Java and is especially useful in containerized environments.
Expand Down Expand Up @@ -135,3 +140,19 @@ Before starting the JVM, create and populate the configuration file and specify
echo otel.service.name=my-service > my.properties
java -Dotel.javaagent.configuration-file=my.properties ...
```

## Agent logging

The EDOT Java agent provides the ability to control the agent log verbosity by setting the log level with the `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` configuration option (`INFO` by default).

The following log levels are supported: `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, `FATAL` and `OFF`.

For [troubleshooting](./troubleshooting.md#agent-debug-logging), the `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=DEBUG` is a recommended alternative to `OTEL_JAVAAGENT_DEBUG=true` as it provides span information in JSON format.

This feature relies on the `OTEL_JAVAAGENT_LOGGING` configuration option to be set to `elastic` (default), the `simple` value from upstream is not supported.

Setting `OTEL_JAVAAGENT_LOGGING=none` or `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=OFF` disables agent logging feature.

Setting `OTEL_JAVAAGENT_LOGGING=application` will disable EDOT agent logging feature and attempt to use the application logger.
As [documented here in the upstream documentation](https://opentelemetry.io/docs/zero-code/java/agent/configuration/#java-agent-logging-output),
support for this depends on the application and logging libraries used.
40 changes: 33 additions & 7 deletions docs/reference/edot-sdks/java/troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,42 @@ Determine if the issue is related to the agent by following these steps:
1. Start the application with no agent and see if the issue is not present. Observe if the issue is again present when restarting with the agent.
2. Check end-to-end connectivity without the agent by running one or more of the example apps in [elastic-otel-java](https://github.com/elastic/elastic-otel-java/blob/main/examples/troubleshooting/README.md). These use the OpenTelemetry SDK rather than the auto-instrumentation. They can confirm that the issue is specific to the Java agent or can otherwise identify that the issue is caused by something else.

## Agent debug output
## Agent debug logging

Turn on debug output with `-Dotel.javaagent.debug=true` or by setting the `OTEL_JAVAAGENT_DEBUG` environment variable to `true`. Turning debug output on might add noticeable overhead. A lower overhead option is to set the `otel.traces.exporter` JVM property or the `OTEL_TRACES_EXPORTER` environment variable to the value `otlp,logging-otlp`: this configures the agent to send the traces both to the Collector and to also print it to the standard output in JSON format.
As debugging output is verbose and might produce noticeable overhead on the application, follow one of these strategies when you need logging:

After debug is activated, look for:
- In case of a technical issue or exception with the agent, use [agent debugging](#agent-debugging).
- If you need details on the captured data, use [per-signal debugging](#per-signal-debugging).

- Errors and exceptions.
- For the expected traces or metrics, or the lack of them. Perhaps the [technology isn't instrumented](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md).
In case of missing data, check first that the technology used in the application is supported in [upstream OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md) and in [EDOT Java](supported-technologies.md).

### Agent debugging

To turn on agent debug logging you can either:

- Set the `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` environment variable or the `elastic.otel.javaagent.log_level` JVM system property to `debug`.
- Set the `OTEL_JAVAAGENT_DEBUG` environment variable or the `otel.javaagent.debug` JVM system property to `true`

Both options require a JVM restart.

The `otel.javaagent.debug` / `OTEL_JAVAAGENT_DEBUG` configuration options are inherited from the upstream
agent. Setting them to `true` also produce span information in plain text format.

When `elastic.otel.javaagent.log_level` or `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` are set to `debug`, the span information is included in JSON format.

If only captured data details are needed, [per-signal debugging](#per-signal-debugging) is a lighter alternative.

### Per-signal debugging

Each supported signal can be logged independently. This allows limiting the amount of captured data and reducing the overhead compared
to [agent debugging](#agent-debugging).

This is configured through the `OTEL_{SIGNAL}_EXPORTER` environment variable or `otel.{signal}.exporter` JVM system property
from the [OpenTelemetry SDK](https://opentelemetry.io/docs/languages/java/configuration/#properties-exporters) by adding any of the following exporters to the default `otlp` value:
- `otlp,logging-otlp`: JSON logging (recommended)
- `otlp,logging`: plain text logging

Both options require a JVM restart.

## Access or modification of application code

Expand Down Expand Up @@ -133,5 +161,3 @@ Updates of the OpenTelemetry API/SDK in the application and the EDOT Java agent
### How to update

Updating EDOT Java agent is done by replacing the agent binary `.jar` that has been [added during setup](./setup/index.md).


Loading