Skip to content

Commit ad5ee38

Browse files
theletterfcolleenmcginnis
theletterf
and
colleenmcginnis
authored
Add reference docs (#386)
* Add docs * Add crosslink to troubleshooting * Update docs/reference/toc.yml Co-authored-by: Colleen McGinnis <[email protected]> * Move files to folder --------- Co-authored-by: Colleen McGinnis <[email protected]>
1 parent ac09080 commit ad5ee38

File tree

11 files changed

+840
-1
lines changed

11 files changed

+840
-1
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,5 @@ tests/*-junit.xml
3535
.aws
3636
.arn-file.md
3737
__pycache__
38+
.artifacts
39+
docs/.artifacts

‎docs/docset.yml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,23 @@ project: 'EDOT Python release notes'
22
cross_links:
33
- docs-content
44
- opentelemetry
5+
- elastic-agent
6+
- apm-agent-python
57
toc:
68
- toc: release-notes
9+
- toc: reference/edot-python
710
subs:
11+
motlp: Elastic Cloud Managed OTLP Endpoint
12+
edot: Elastic Distribution of OpenTelemetry
13+
ecloud: "Elastic Cloud"
14+
edot-cf: "EDOT Cloud Forwarder"
15+
ech: "Elastic Cloud Hosted"
16+
ess: "Elasticsearch Service"
17+
ece: "Elastic Cloud Enterprise"
18+
serverless-full: "Elastic Cloud Serverless"
819
agent: "Elastic Agent"
920
agents: "Elastic Agents"
10-
edot: "Elastic Distribution of OpenTelemetry"
21+
stack: "Elastic Stack"
22+
es: "Elasticsearch"
23+
kib: "Kibana"
24+
ls: "Logstash"

‎docs/reference/edot-python/configuration.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
navigation_title: Configuration
3+
description: Configure the Elastic Distribution of OpenTelemetry Python (EDOT Python) to send data to Elastic.
4+
applies_to:
5+
stack:
6+
serverless:
7+
observability:
8+
product:
9+
edot_python: ga
10+
products:
11+
- id: cloud-serverless
12+
- id: observability
13+
- id: edot-sdk
14+
---
15+
16+
# Configure the EDOT Python agent
17+
18+
Configure the {{edot}} Python (EDOT Python) to send data to Elastic.
19+
20+
## Configuration method
21+
22+
Configure the OpenTelemetry SDK through the mechanisms [documented on the OpenTelemetry website](https://opentelemetry.io/docs/zero-code/python/configuration/). EDOT Python is typically configured with `OTEL_*` environment variables defined by the OpenTelemetry spec. For example:
23+
24+
```sh
25+
export OTEL_RESOURCE_ATTRIBUTES=service.name=<app-name>,deployment.environment.name=<env-name>
26+
export OTEL_EXPORTER_OTLP_ENDPOINT=https://my-deployment.ingest.us-west1.gcp.cloud.es.io
27+
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey P....l"
28+
opentelemetry-instrument <command to start your service>
29+
```
30+
31+
## Configuration options
32+
33+
Because the {{edot}} Python is an extension of OpenTelemetry Python, it supports both:
34+
35+
* [General OpenTelemetry configuration options](#opentelemetry-configuration-options)
36+
* [Specific configuration options that are only available in EDOT Python](#configuration-options-only-available-in-edot-python)
37+
38+
## Central configuration
39+
40+
```{applies_to}
41+
serverless: unavailable
42+
stack: preview 9.1
43+
product:
44+
edot_python: preview 1.4.0
45+
```
46+
47+
APM Agent Central Configuration lets you configure EDOT Python instances remotely, see [Central configuration docs](opentelemetry://reference/central-configuration.md) for more details.
48+
49+
### Turn on central configuration
50+
51+
To activate central configuration, set the `ELASTIC_OTEL_OPAMP_ENDPOINT` environment variable to the OpAMP server endpoint.
52+
53+
```sh
54+
export ELASTIC_OTEL_OPAMP_ENDPOINT=http://localhost:4320/v1/opamp
55+
```
56+
57+
To deactivate central configuration, remove the `ELASTIC_OTEL_OPAMP_ENDPOINT` environment variable and restart the instrumented application.
58+
59+
### Central configuration settings
60+
61+
You can modify the following settings for EDOT Python through APM Agent Central Configuration:
62+
63+
| Settings | Description | Type | Versions |
64+
|---------------|----------------------------------------------|---------|---------|
65+
| Logging level | Configure EDOT Python agent logging level. | Dynamic | {applies_to}`stack: preview 9.1` <br> {applies_to}`edot_python: preview 1.4.0` |
66+
| Sampling rate | Configure EDOT Python tracing sampling rate. | Dynamic | {applies_to}`stack: preview 9.2` <br> {applies_to}`edot_python: preview 1.7.0` |
67+
68+
Dynamic settings can be changed without having to restart the application.
69+
70+
### OpenTelemetry configuration options
71+
72+
EDOT Python supports all configuration options listed in the [OpenTelemetry General SDK Configuration documentation](https://opentelemetry.io/docs/languages/sdk-configuration/general/) and [OpenTelemetry Python](https://opentelemetry.io/docs/languages/python).
73+
74+
#### Logs
75+
76+
Instrument Python `logging` module to format and forward logs in OTLP format is turned off by default and gated under a configuration environment variable:
77+
78+
```sh
79+
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
80+
```
81+
82+
:::{note}
83+
Turning this on will make any call to [logging.basicConfig](https://docs.python.org/3/library/logging.html#logging.basicConfig) from your application a no-op.
84+
:::
85+
86+
#### Differences from OpenTelemetry Python
87+
88+
EDOT Python uses different defaults than OpenTelemetry Python for the following configuration options:
89+
90+
| Option | EDOT Python default | OpenTelemetry Python default |
91+
|---|---|---|
92+
| `OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` | `process_runtime,os,otel,telemetry_distro,service_instance,containerid,_gcp,aws_ec2,aws_ecs,aws_elastic_beanstalk,azure_app_service,azure_vm` | `otel` |
93+
| `OTEL_METRICS_EXEMPLAR_FILTER` | `always_off` | `trace_based` |
94+
| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | `DELTA` | `CUMULATIVE` |
95+
| `OTEL_TRACES_SAMPLER` | `parentbased_traceidratio` | `parentbased_always_on` |
96+
| `OTEL_TRACES_SAMPLER_ARG` | `1.0` | |
97+
98+
:::{note}
99+
`OTEL_EXPERIMENTAL_RESOURCE_DETECTORS` cloud resource detectors are dynamically set. When running in a Kubernetes Pod it will be set to `process_runtime,os,otel,telemetry_distro,service_instance,_gcp,aws_eks`.
100+
:::
101+
102+
103+
### Configuration options only available in EDOT Python
104+
105+
`ELASTIC_OTEL_` options are specific to Elastic and will always live in EDOT Python include the following.
106+
107+
| Option(s) | Default | Description |
108+
|---|---|---|
109+
| `ELASTIC_OTEL_SYSTEM_METRICS_ENABLED` | `false` | When sets to `true`, sends *system namespace* metrics. |
110+
111+
## LLM settings
112+
113+
LLM instrumentations implement the following configuration options:
114+
115+
| Option | default | description |
116+
|-------------------------------------------------------|---------|:--------------------------|
117+
| `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | `false`| If set to `true`, enables the capturing of request and response content in the log events outputted by the agent.
118+
119+
120+
## Prevent logs export
121+
122+
To prevent logs from being exported, set `OTEL_LOGS_EXPORTER` to `none`. However, application logs might still be gathered and exported by the Collector through the `filelog` receiver.
123+
124+
To prevent application logs from being collected and exported by the Collector, refer to [Exclude paths from logs collection](elastic-agent://reference/edot-collector/config/configure-logs-collection.md#exclude-logs-paths).

‎docs/reference/edot-python/index.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
navigation_title: EDOT Python
3+
description: The Elastic Distribution of OpenTelemetry Python (EDOT Python) is a customized version of OpenTelemetry Python.
4+
applies_to:
5+
stack:
6+
serverless:
7+
observability:
8+
product:
9+
edot_python: ga
10+
products:
11+
- id: cloud-serverless
12+
- id: observability
13+
- id: edot-sdk
14+
---
15+
16+
# Elastic Distribution of OpenTelemetry Python
17+
18+
The [{{edot}} (EDOT) Python](https://github.com/elastic/elastic-otel-python) is a customized version of [OpenTelemetry Python](https://opentelemetry.io/docs/languages/python), configured for the best experience with Elastic Observability.
19+
20+
Use EDOT Python to start the OpenTelemetry SDK with your Python application, and automatically capture tracing data, performance metrics, and logs. Traces, metrics, and logs can be sent to any OpenTelemetry Protocol (OTLP) Collector you choose.
21+
22+
A goal of this distribution is to avoid introducing proprietary concepts in addition to those defined by the wider OpenTelemetry community. For any additional features introduced, Elastic aims at contributing them back to the OpenTelemetry project.
23+
24+
## Features
25+
26+
In addition to all the features of the OpenTelemetry Python agent, with EDOT Python you have access to the following:
27+
28+
* Improvements and bug fixes contributed by the Elastic team before the changes are available in OpenTelemetry repositories.
29+
* Optional features that can enhance OpenTelemetry data that is being sent to Elastic.
30+
* Elastic-specific processors that ensure optimal compatibility when exporting OpenTelemetry signal data to an Elastic backend like an Elastic Observability deployment.
31+
* Preconfigured collection of tracing and metrics signals, applying some opinionated defaults, such as which sources are collected by default.
32+
* Compatibility with APM Agent Central Configuration to modify the settings of the EDOT Python agent without having to restart the application.
33+
34+
Follow the step-by-step instructions in [Setup](/reference/edot-python/setup/index.md) to get started.
35+
36+
## Release notes
37+
38+
For the latest release notes, including known issues, deprecations, and breaking changes, refer to [EDOT Python release notes](/release-notes/index.md)

‎docs/reference/edot-python/migration.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
---
2+
navigation_title: Migration
3+
description: Migrate from the Elastic APM Python agent to the Elastic Distribution of OpenTelemetry Python (EDOT Python).
4+
applies_to:
5+
stack:
6+
serverless:
7+
observability:
8+
product:
9+
edot_python: ga
10+
products:
11+
- id: cloud-serverless
12+
- id: observability
13+
- id: edot-sdk
14+
- id: apm-agent
15+
---
16+
17+
# Migrate to EDOT Python from the Elastic APM Python agent
18+
19+
Learn the differences between the [Elastic APM Python agent](apm-agent-python://reference/index.md) and the {{edot}} Python (EDOT Python).
20+
21+
Follow the steps to migrate your instrumentation and settings. For step-by-step instructions on setting up EDOT Python refer to [Setup](/reference/edot-python/setup/index.md).
22+
23+
## Migration steps
24+
25+
Follow these steps to migrate:
26+
27+
1. Remove any configuration and setup code needed by Elastic APM Python Agent from your application source code.
28+
2. Migrate any usage of Elastic APM Python Agent API to manual instrumentation with OpenTelemetry API in the application source code.
29+
3. Follow the [setup documentation](setup/index.md) on to install and configure EDOT Python.
30+
31+
## Configuration mapping
32+
33+
The following are Elastic APM Python agent settings that you can migrate to EDOT Python.
34+
35+
### `api_key`
36+
37+
The Elastic [`api_key`](apm-agent-python://reference/configuration.md#config-api-key) option corresponds to the OpenTelemetry [OTEL_EXPORTER_OTLP_HEADERS](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_headers) option.
38+
39+
For example: `OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_api_key"`.
40+
41+
### `enabled`
42+
43+
The Elastic [`enabled`](apm-agent-python://reference/configuration.md#config-enabled) option corresponds to the OpenTelemetry [OTEL_SDK_DISABLED](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration) option.
44+
45+
### `environment`
46+
47+
The Elastic [`environment`](apm-agent-python://reference/configuration.md#config-environment) option corresponds to setting the `deployment.environment.name` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes).
48+
49+
For example: `OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=testing`.
50+
51+
### `global_labels`
52+
53+
The Elastic [`global_labels`](apm-agent-python://reference/configuration.md#config-global_labels) option corresponds to adding `key=value` comma separated pairs in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes).
54+
55+
For example: `OTEL_RESOURCE_ATTRIBUTES=alice=first,bob=second`. Such labels will result in labels.key=value attributes on the server, e.g. labels.alice=first
56+
57+
### `metrics_interval`
58+
59+
The Elastic [`metrics_interval`](apm-agent-python://reference/configuration.md#config-metrics_interval) corresponds to the OpenTelemetry [OTEL_METRIC_EXPORT_INTERVAL](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#periodic-exporting-metricreader) option.
60+
61+
For example: `OTEL_METRIC_EXPORT_INTERVAL=30000`.
62+
63+
### `secret_token`
64+
65+
The Elastic [`secret_token`](apm-agent-python://reference/configuration.md#config-secret-token) option corresponds to the OpenTelemetry [OTEL_EXPORTER_OTLP_HEADERS](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_headers) option.
66+
67+
For example: `OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_apm_secret_token"`.
68+
69+
### `server_url`
70+
71+
The Elastic [`server_url`](apm-agent-python://reference/configuration.md#config-server-url) option corresponds to the OpenTelemetry [`OTEL_EXPORTER_OTLP_ENDPOINT`](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_endpoint) option.
72+
73+
### `service_name`
74+
75+
The Elastic [`service_name`](apm-agent-python://reference/configuration.md#config-service-name) option corresponds to the OpenTelemetry [OTEL_SERVICE_NAME](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_service_name) option.
76+
77+
You can also set the service name using [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes).
78+
79+
For example: `OTEL_RESOURCE_ATTRIBUTES=service.name=myservice`. If `OTEL_SERVICE_NAME` is set, it takes precedence over the resource attribute.
80+
81+
### `service_version`
82+
83+
The Elastic [`service_version`](apm-agent-python://reference/configuration.md#config-service-version) option corresponds to setting the `service.version` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes).
84+
85+
For example: `OTEL_RESOURCE_ATTRIBUTES=service.version=1.2.3`.
86+
87+
## Performance overhead
88+
89+
Evaluate the [differences in performance overhead](/reference/edot-python/overhead.md) between EDOT Python and Elastic APM Python agent.
90+
91+
## Limitations
92+
93+
The following limitations apply when migrating to EDOT Python.
94+
95+
### Central and dynamic configuration
96+
97+
You can manage EDOT Python configurations through the [central configuration feature](docs-content://solutions/observability/apm/apm-agent-central-configuration.md) in the Applications UI.
98+
99+
Refer to [Central configuration](opentelemetry://reference/central-configuration.md) for more information.
100+
101+
### AWS Lambda
102+
103+
A custom lambda layer for the {{edot}} Python is not currently available. Refer to the [Lambda Auto-Instrumentation](https://opentelemetry.io/docs/faas/lambda-auto-instrument/).
104+
105+
### Missing instrumentations
106+
107+
The following libraries are currently missing an OpenTelemetry equivalent:
108+
109+
- Azure storage and Azure queue
110+
- `aiobotocore`
111+
- `aiomysql`
112+
- `aioredis`
113+
- `Graphene`
114+
- `httplib2`
115+
- `pylibmc`
116+
- `pyodbc`
117+
- `python-memcached`
118+
- `Sanic`
119+
- `zlib`
120+
121+
### Integration with structured logging
122+
123+
EDOT Python lacks a [structlog integration](apm-agent-python://reference/logs.md#structlog) at the moment.
124+
125+
### Span compression
126+
127+
EDOT Python does not implement [span compression](docs-content://solutions/observability/apm/spans.md#apm-spans-span-compression).
128+
129+
### Breakdown metrics
130+
131+
EDOT Python is not sending metrics that power the [Breakdown metrics](docs-content://solutions/observability/apm/metrics.md#_breakdown_metrics).
132+
133+
## Troubleshooting
134+
135+
If you're encountering issues during migration, refer to the [EDOT Python troubleshooting guide](docs-content://troubleshoot/ingest/opentelemetry/edot-sdks/python/index.md).

‎docs/reference/edot-python/overhead.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
navigation_title: Performance overhead
3+
description: This page explains the performance considerations when instrumenting Python applications with the Elastic Distribution of OpenTelemetry SDK, including impact analysis and mitigation techniques.
4+
applies_to:
5+
stack:
6+
serverless:
7+
observability:
8+
product:
9+
edot_python: ga
10+
products:
11+
- id: cloud-serverless
12+
- id: observability
13+
- id: edot-sdk
14+
---
15+
16+
# Performance overhead of the EDOT SDK for Python
17+
18+
This page explains the performance considerations when instrumenting Python applications with the Elastic Distribution of OpenTelemetry SDK, including impact analysis and mitigation techniques.
19+
20+
While designed to have minimal performance overhead, the EDOT Java agent, like any instrumentation agent, executes within the application process and thus has a small influence on the application performance.
21+
22+
This performance overhead depends on the application's technical architecture, its configuration and environment, and the load. These factors are not easy to reproduce on their own, and all applications are different, so it is not possible to provide a simple answer.
23+
24+
## Benchmark
25+
26+
The following numbers are only provided as indicators, and you should not attempt to extrapolate them. Use them as a framework to evaluate and measure the overhead on your applications.
27+
28+
The following table compares the response times of a sample web application without an agent, with Elastic APM Python Agent and with EDOT Python Agent in two situations: without data loaded and serialized to measure the minimal overhead of agents and with some data loaded and then serialized to provide a more common scenario.
29+
30+
| | No agent | EDOT Python agent | Elastic APM Python agent |
31+
|-----------------------------------|-----------|-----------------------------|--------------------------|
32+
| No data: Time taken for tests | 1.277 s | 2.215 s | 2.313 s |
33+
| Sample data: Time taken for tests | 4.546 s | 6.401 s | 6.159 s |

0 commit comments

Comments
 (0)