You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
: Set a default topic to use for events sent by {{agent}} to the Kafka output.
130
+
: Set the default Kafka topic used for events sent by {{agent}}.
131
131
132
132
You can set a static topic, for example `elastic-agent`, or you can choose to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
133
133
134
134
* `data_stream.type`
135
135
* `data_stream.dataset`
136
136
* `data_stream.namespace`
137
137
* `@timestamp`
138
-
* `event-dataset`
138
+
* `event.dataset`
139
139
140
-
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name.
140
+
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name. For example, you can use the `fields.kafka_topic` custom field to set a dynamic topic for each event.
141
141
142
142
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to any integration policies on your {{fleet}}-managed {{agents}}.
143
143
144
-
For example, the following `add_fields` processor creates a dynamic topic value by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
144
+
For example, the following `add_fields` processor creates a dynamic topic value for the `fields.kafka_topic` field by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
1. Depending on the values of the data stream fields, this generates topic names such as `logs-nginx.access-production` or `metrics-system.cpu-staging` as the value of the custom `kafka_topic` field.
Copy file name to clipboardExpand all lines: reference/fleet/kafka-output.md
+18-8Lines changed: 18 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -164,25 +164,35 @@ Use these options to set the Kafka topic for each {{agent}} event.
164
164
`topic`$$$kafka-topic-setting$$$
165
165
: The default Kafka topic used for produced events.
166
166
167
-
You can set a static topic, for example `elastic-agent`, or you can choose to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
167
+
You can set a static topic, for example `elastic-agent`, or you can use a format string to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
168
168
169
169
* `data_stream.type`
170
170
* `data_stream.dataset`
171
171
* `data_stream.namespace`
172
172
* `@timestamp`
173
-
* `event-dataset`
173
+
* `event.dataset`
174
+
175
+
For example:
176
+
177
+
```yaml
178
+
topic: '${data_stream.type}'
179
+
```
174
180
175
-
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name.
181
+
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name. For example, this configuration uses the `fields.kafka_topic` custom field to set the topic for each event:
182
+
183
+
```yaml
184
+
topic: '${fields.kafka_topic}'
185
+
```
176
186
177
-
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to the input configuration settings of your standalone {{agent}}.
187
+
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to the input configuration settings of your standalone {{agent}}.
178
188
179
-
For example, the following `add_fields` processor creates a dynamic topic value by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
189
+
For example, the following `add_fields` processor creates a dynamic topic value for the `fields.kafka_topic` field by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
1. Depending on the values of the data stream fields, this generates topic names such as `logs-nginx.access-production` or `metrics-system.cpu-staging` as the value of the custom `kafka_topic` field.
0 commit comments