Skip to content

Commit 6a3da73

Browse files
committed
Update the topic description of the Kafka output docs
1 parent ac8313b commit 6a3da73

File tree

2 files changed

+25
-15
lines changed

2 files changed

+25
-15
lines changed

reference/fleet/kafka-output-settings.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -127,27 +127,27 @@ The number of partitions created is set automatically by the Kafka broker based
127127
Use this option to set the Kafka topic for each {{agent}} event.
128128

129129
**Default topic** $$$kafka-output-topics-default$$$
130-
: Set a default topic to use for events sent by {{agent}} to the Kafka output.
130+
: Set the default Kafka topic used for events sent by {{agent}}.
131131

132132
You can set a static topic, for example `elastic-agent`, or you can choose to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
133133

134134
* `data_stream.type`
135135
* `data_stream.dataset`
136136
* `data_stream.namespace`
137137
* `@timestamp`
138-
* `event-dataset`
138+
* `event.dataset`
139139

140-
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name.
140+
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name. For example, you can use the `fields.kafka_topic` custom field to set a dynamic topic for each event.
141141

142142
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to any integration policies on your {{fleet}}-managed {{agents}}.
143143

144-
For example, the following `add_fields` processor creates a dynamic topic value by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
144+
For example, the following `add_fields` processor creates a dynamic topic value for the `fields.kafka_topic` field by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
145145

146146
```yaml
147147
- add_fields:
148-
target: ''
149-
fields:
150-
kafka_topic: '${data_stream.type}-${data_stream.dataset}-${data_stream.namespace}' <1>
148+
target: ''
149+
fields:
150+
kafka_topic: '${data_stream.type}-${data_stream.dataset}-${data_stream.namespace}' <1>
151151
```
152152
1. Depending on the values of the data stream fields, this generates topic names such as `logs-nginx.access-production` or `metrics-system.cpu-staging` as the value of the custom `kafka_topic` field.
153153

reference/fleet/kafka-output.md

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -164,25 +164,35 @@ Use these options to set the Kafka topic for each {{agent}} event.
164164
`topic` $$$kafka-topic-setting$$$
165165
: The default Kafka topic used for produced events.
166166

167-
You can set a static topic, for example `elastic-agent`, or you can choose to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
167+
You can set a static topic, for example `elastic-agent`, or you can use a format string to set a topic dynamically based on an [Elastic Common Schema (ECS)](ecs://reference/index.md) field. Available fields include:
168168

169169
* `data_stream.type`
170170
* `data_stream.dataset`
171171
* `data_stream.namespace`
172172
* `@timestamp`
173-
* `event-dataset`
173+
* `event.dataset`
174+
175+
For example:
176+
177+
```yaml
178+
topic: '${data_stream.type}'
179+
```
174180

175-
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name.
181+
You can also set a custom field. This is useful if you need to construct a more complex or structured topic name. For example, this configuration uses the `fields.kafka_topic` custom field to set the topic for each event:
182+
183+
```yaml
184+
topic: '${fields.kafka_topic}'
185+
```
176186

177-
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to the input configuration settings of your standalone {{agent}}.
187+
To set a dynamic topic value for outputting {{agent}} data to Kafka, you can add the [`add_fields` processor](/reference/fleet/add_fields-processor.md) to the input configuration settings of your standalone {{agent}}.
178188

179-
For example, the following `add_fields` processor creates a dynamic topic value by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
189+
For example, the following `add_fields` processor creates a dynamic topic value for the `fields.kafka_topic` field by interpolating multiple [data stream fields](ecs://reference/ecs-data_stream.md):
180190

181191
```yaml
182192
- add_fields:
183-
target: ''
184-
fields:
185-
kafka_topic: '${data_stream.type}-${data_stream.dataset}-${data_stream.namespace}' <1>
193+
target: ''
194+
fields:
195+
kafka_topic: '${data_stream.type}-${data_stream.dataset}-${data_stream.namespace}' <1>
186196
```
187197
1. Depending on the values of the data stream fields, this generates topic names such as `logs-nginx.access-production` or `metrics-system.cpu-staging` as the value of the custom `kafka_topic` field.
188198

0 commit comments

Comments
 (0)