@@ -112,7 +112,29 @@ fluent-bit -i cpu -t cpu -o es -p Host=192.168.2.3 -p Port=9200 \
112
112
113
113
In your main configuration file append the following ` Input ` and ` Output ` sections.
114
114
115
- ``` python
115
+ % tabs %}
116
+ {% tab title="fluent-bit.yaml" %}
117
+
118
+ ``` yaml
119
+ pipeline :
120
+ inputs :
121
+ - name : cpu
122
+ tag : cpu
123
+
124
+ outputs :
125
+ - name : es
126
+ match : ' *'
127
+ host : 192.168.2.3
128
+ port : 9200
129
+ index : my_index
130
+ type : my_type
131
+ ` ` `
132
+
133
+ {% endtab %}
134
+
135
+ {% tab title="fluent-bit.conf" %}
136
+
137
+ ` ` ` text
116
138
[INPUT]
117
139
Name cpu
118
140
Tag cpu
@@ -126,7 +148,8 @@ In your main configuration file append the following `Input` and `Output` sectio
126
148
Type my_type
127
149
```
128
150
129
- ![ example configuration visualization from Calyptia] ( ../../.gitbook/assets/image%20%282%29.png )
151
+ {% endtab %}
152
+ {% endtabs %}
130
153
131
154
## About Elasticsearch field names
132
155
@@ -158,7 +181,31 @@ See [details](https://github.com/fluent/fluent-bit-docs/tree/43c4fe134611da471e7
158
181
159
182
Example configuration:
160
183
161
- ``` text copy
184
+ % tabs %}
185
+ {% tab title="fluent-bit.yaml" %}
186
+
187
+ ``` yaml
188
+ pipeline :
189
+ inputs :
190
+ ...
191
+
192
+ outputs :
193
+ - name : es
194
+ match : ' *'
195
+ host : vpc-test-domain-ke7thhzoo7jawsrhmm6mb7ite7y.us-west-2.es.amazonaws.com
196
+ port : 443
197
+ index : my_index
198
+ type : my_type
199
+ aws_auth : on
200
+ aws_region : us-west-2
201
+ tls : on
202
+ ` ` `
203
+
204
+ {% endtab %}
205
+
206
+ {% tab title="fluent-bit.conf" %}
207
+
208
+ ` ` ` text
162
209
[OUTPUT]
163
210
Name es
164
211
Match *
@@ -171,6 +218,9 @@ Example configuration:
171
218
tls On
172
219
```
173
220
221
+ {% endtab %}
222
+ {% endtabs %}
223
+
174
224
Be aware that the ` Port ` is set to ` 443 ` , ` tls ` is enabled, and ` AWS_Region ` is set.
175
225
176
226
### Use Fluent Bit with Elastic Cloud
@@ -184,7 +234,29 @@ to the
184
234
185
235
Example configuration:
186
236
187
- ``` text copy
237
+ % tabs %}
238
+ {% tab title="fluent-bit.yaml" %}
239
+
240
+ ``` yaml
241
+ pipeline :
242
+ inputs :
243
+ ...
244
+
245
+ outputs :
246
+ - name : es
247
+ include_tag_key : true
248
+ tag_key : tags
249
+ tls : on
250
+ tls.verify : off
251
+ cloud_id : ' elastic-obs-deployment:ZXVybxxxxxxxxxxxg=='
252
+ cloud_auth : ' elastic:2vxxxxxxxxYV'
253
+ ` ` `
254
+
255
+ {% endtab %}
256
+
257
+ {% tab title="fluent-bit.conf" %}
258
+
259
+ ` ` ` text
188
260
[OUTPUT]
189
261
Name es
190
262
Include_Tag_Key true
@@ -196,6 +268,9 @@ Example configuration:
196
268
cloud_auth elastic:2vxxxxxxxxYV
197
269
```
198
270
271
+ {% endtab %}
272
+ {% endtabs %}
273
+
199
274
In Elastic Cloud version 8 and great, the type option must be removed by setting
200
275
` Suppress_Type_Name On ` .
201
276
@@ -221,6 +296,30 @@ Rejecting mapping update to [products] as the final mapping would have more than
221
296
222
297
This means that you can't set up your configuration like the following:.
223
298
299
+ % tabs %}
300
+ {% tab title="fluent-bit.yaml" %}
301
+
302
+ ``` yaml
303
+ pipeline :
304
+ inputs :
305
+ ...
306
+
307
+ outputs :
308
+ - name : es
309
+ match : ' foo.*'
310
+ index : search
311
+ type : type1
312
+
313
+ - name : es
314
+ match : ' bar.*'
315
+ index : search
316
+ type : type2
317
+ ` ` `
318
+
319
+ {% endtab %}
320
+
321
+ {% tab title="fluent-bit.conf" %}
322
+
224
323
` ` ` text
225
324
[OUTPUT]
226
325
Name es
@@ -235,6 +334,9 @@ This means that you can't set up your configuration like the following:.
235
334
Type type2
236
335
```
237
336
337
+ {% endtab %}
338
+ {% endtabs %}
339
+
238
340
For details, read [ the official blog post on that issue] ( https://www.elastic.co/guide/en/elasticsearch/reference/6.7/removal-of-types.html ) .
239
341
240
342
### Mapping type names can't start with underscores (` _ ` )
@@ -250,7 +352,31 @@ This doesn't work in Elasticsearch versions 5.6 through 6.1
250
352
Ensure you set an explicit map such as ` doc ` or ` flb_type ` in the configuration,
251
353
as seen on the last line:
252
354
253
- ``` text copy
355
+ % tabs %}
356
+ {% tab title="fluent-bit.yaml" %}
357
+
358
+ ``` yaml
359
+ pipeline :
360
+ inputs :
361
+ ...
362
+
363
+ outputs :
364
+ - name : es
365
+ match : ' *'
366
+ host : vpc-test-domain-ke7thhzoo7jawsrhmm6mb7ite7y.us-west-2.es.amazonaws.com
367
+ port : 443
368
+ index : my_index
369
+ aws_auth : on
370
+ aws_region : us-west-2
371
+ tls : on
372
+ type : doc
373
+ ` ` `
374
+
375
+ {% endtab %}
376
+
377
+ {% tab title="fluent-bit.conf" %}
378
+
379
+ ` ` ` text
254
380
[OUTPUT]
255
381
Name es
256
382
Match *
@@ -263,10 +389,13 @@ as seen on the last line:
263
389
Type doc
264
390
```
265
391
392
+ {% endtab %}
393
+ {% endtabs %}
394
+
266
395
### Validation failures
267
396
268
397
In Fluent Bit v1.8.2 and greater, Fluent Bit started using ` create ` method (instead
269
- of ` index ` ) for data submission. This makes Fluent Bit compatible with Datastream,
398
+ of ` index ` ) for data submission. This makes Fluent Bit compatible with ` Datastream ` ,
270
399
introduced in Elasticsearch 7.9. You might see errors like:
271
400
272
401
``` text
@@ -277,19 +406,62 @@ If you see `action_request_validation_exception` errors on your pipeline with
277
406
Fluent Bit versions greater than v1.8.2, correct them by turning on ` Generate_ID `
278
407
as follows:
279
408
280
- ``` text copy
409
+ % tabs %}
410
+ {% tab title="fluent-bit.yaml" %}
411
+
412
+ ``` yaml
413
+ pipeline :
414
+ inputs :
415
+ ...
416
+
417
+ outputs :
418
+ - name : es
419
+ match : ' *'
420
+ host : 192.168.12.1
421
+ generate_id : on
422
+ ` ` `
423
+
424
+ {% endtab %}
425
+
426
+ {% tab title="fluent-bit.conf" %}
427
+
428
+ ` ` ` text
281
429
[OUTPUT]
282
430
Name es
283
431
Match *
284
432
Host 192.168.12.1
285
433
Generate_ID on
286
434
```
287
435
436
+ {% endtab %}
437
+ {% endtabs %}
438
+
288
439
### ` Logstash_Prefix_Key `
289
440
290
441
The following snippet demonstrates using the namespace name as extracted by the
291
442
` kubernetes ` filter as ` logstash ` prefix:
292
443
444
+ % tabs %}
445
+ {% tab title="fluent-bit.yaml" %}
446
+
447
+ ``` yaml
448
+ pipeline :
449
+ inputs :
450
+ ...
451
+
452
+ outputs :
453
+ - name : es
454
+ match : ' *'
455
+ # ...
456
+ logstash_prefix : logstash
457
+ logstash_prefix_key : $kubernetes['namespace_name']
458
+ # ...
459
+ ```
460
+
461
+ {% endtab %}
462
+
463
+ {% tab title="fluent-bit.conf" %}
464
+
293
465
``` text
294
466
[OUTPUT]
295
467
Name es
@@ -300,5 +472,8 @@ The following snippet demonstrates using the namespace name as extracted by the
300
472
# ...
301
473
```
302
474
475
+ {% endtab %}
476
+ {% endtabs %}
477
+
303
478
For records that don't have the field ` kubernetes.namespace_name ` , the default prefix
304
- ` logstash ` will be used.
479
+ ` logstash ` will be used.
0 commit comments