Skip to content
Original file line number Diff line number Diff line change
Expand Up @@ -131,15 +131,32 @@ The `elasticsearch.index` attribute is removed from the final document if it exi

## Performance and batching

The exporter supports both internal batching and OpenTelemetry's standard `sending_queue` configuration:
### Using sending queue

The {{es}} exporter supports the `sending_queue` setting, which supports both queueing and batching. The sending queue is deactivated by default.

You can turn on the sending queue by setting `sending_queue::enabled` to `true`:

```yaml subs=true
exporters:
elasticsearch:
endpoint: https://elasticsearch:9200
sending_queue:
enabled: true
```

### Internal batching (default)

By default, the exporter performs its own buffering and batching, as configured through the `flush` setting, unless the `sending_queue` and `batcher` settings are defined.
By default, the exporter performs its own buffering and batching, as configured through the `flush` setting, unless the `sending_queue::batch` or the `batcher` settings are defined. In that case, batching is controlled by either of the two settings, depending on the version.

### Using sending queue
### Custom batching

The Elasticsearch exporter supports the `sending_queue` setting, which supports both queueing and batching. However, the sending queue is currently deactivated by default. You can turn on the sending queue by setting `sending_queue` to true. Batching support in sending queue is also deactivated by default and can be turned on by defining `sending_queue::batch`. For example:
::::{applies-switch}

:::{applies-item} stack: ga 9.2
Batching support in sending queue is deactivated by default. To turn it on, enable sending queue and define `sending_queue::batch`.

For example:

```yaml subs=true
exporters:
Expand All @@ -148,11 +165,37 @@ exporters:
sending_queue:
enabled: true
batch:
enabled: true
min_size: 1000
max_size: 10000
timeout: 5s
```
:::

:::{applies-item} stack: ga 9.0, deprecated 9.2

Batching can be enabled and configured with the `batcher` section, using [common `batcher` settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/internal/queue_sender.go).

- `batcher`:
- `enabled` (default=unset): Enable batching of requests into 1 or more bulk requests. On a batcher flush, it is possible for a batched request to be translated to more than 1 bulk request due to `flush::bytes`.
- `sizer` (default=items): Unit of `min_size` and `max_size`. Currently supports only "items", in the future will also support "bytes".
- `min_size` (default=5000): Minimum batch size to be exported to Elasticsearch, measured in units according to `batcher::sizer`.
- `max_size` (default=0): Maximum batch size to be exported to Elasticsearch, measured in units according to `batcher::sizer`. To limit bulk request size, configure `flush::bytes` instead. :warning: It is recommended to keep `max_size` as 0 as a non-zero value may lead to broken metrics grouping and indexing rejections.
- `flush_timeout` (default=10s): Maximum time of the oldest item spent inside the batcher buffer, aka "max age of batcher buffer". A batcher flush will happen regardless of the size of content in batcher buffer.

For example:

```yaml subs=true
exporters:
elasticsearch:
endpoint: https://elasticsearch:9200
batcher:
enabled: true
min_size: 1000
max_size: 10000
flush_timeout: 5s
```
:::
::::

## Bulk indexing

Expand Down
Loading