Skip to content

Commit 0896d7a

Browse files
eschabelleschabell
authored
Adding YAML examples, fixing standards for table layouts for output plugins form range N and O. Fixes #1926. (#1930)
* Adding YAML examples and fixing table layout. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding YAML examples to NULL output plugin doc. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding YAML examples and fixing table layout for Observe output plugin. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding back missing original OpenObserve output plugin doc, adding YAML examples, and fixing table layout. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding YAML examples and fixing table layouts for OpenSearch output plugin doc. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding YAML examples and fixing table layouts for OpenTelemetry output plugin doc. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> * Adding YAML examples and fixing table layouts for Oracle Log Analytics output plugin doc. Part of isssue #1926. Signed-off-by: Eric D. Schabell <[email protected]> --------- Signed-off-by: Eric D. Schabell <[email protected]>
1 parent 5610226 commit 0896d7a

File tree

7 files changed

+686
-316
lines changed

7 files changed

+686
-316
lines changed

‎pipeline/outputs/new-relic.md

Lines changed: 56 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -10,111 +10,74 @@ Before to get started with the plugin configuration, make sure to obtain the pro
1010

1111
## Configuration Parameters
1212

13-
| Key | Description | Default |
14-
| :--- | :--- | :--- |
15-
16-
17-
<table>
18-
<thead>
19-
<tr>
20-
<th style="text-align:left">base_uri</th>
21-
<th style="text-align:left">
22-
<p>Full address of New Relic API end-point. By default the value points to
23-
the US end-point.
24-
<br />
25-
</p>
26-
<p>If you want to use the EU end-point you can set this key to the following
27-
value:
28-
<br />
29-
<br /><a href="https://log-api.eu.newrelic.com/log/v1">https://log-api.eu.newrelic.com/log/v1</a>
30-
</p>
31-
</th>
32-
<th style="text-align:left"><a href="https://log-api.newrelic.com/log/v1">https://log-api.newrelic.com/log/v1</a>
33-
</th>
34-
</tr>
35-
</thead>
36-
<tbody></tbody>
37-
</table>
38-
39-
<table>
40-
<thead>
41-
<tr>
42-
<th style="text-align:left">api_key</th>
43-
<th style="text-align:left">
44-
<p>Your key for data ingestion. The API key is also called the ingestion
45-
key, you can get more details on how to generated in the official documentation
46-
<a
47-
href="https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#event-insert-key">here</a>.</p>
48-
<p>From a configuration perspective either an <code>api_key</code> or an <code>license_key</code> is
49-
required. New Relic suggest to use primary the <code>api_key</code>.</p>
50-
</th>
51-
<th style="text-align:left"></th>
52-
</tr>
53-
</thead>
54-
<tbody></tbody>
55-
</table>
56-
57-
<table>
58-
<thead>
59-
<tr>
60-
<th style="text-align:left">license_key</th>
61-
<th style="text-align:left">
62-
<p>Optional authentication parameter for data ingestion.
63-
<br />
64-
</p>
65-
<p>Note that New Relic suggest to use the <code>api_key</code> instead. You
66-
can read more about the License Key <a href="https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/license-key">here</a>.</p>
67-
</th>
68-
<th style="text-align:left"></th>
69-
</tr>
70-
</thead>
71-
<tbody></tbody>
72-
</table>
73-
74-
| compress | Set the compression mechanism for the payload. This option allows two values: `gzip` \(enabled by default\) or `false` to disable compression. | gzip |
75-
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
76-
| :--- | :--- | :--- |
77-
78-
79-
The following configuration example, will emit a dummy example record and ingest it on New Relic. Copy and paste the following content in a file called `newrelic.conf`:
80-
81-
```python
13+
| Key | Description | Default |
14+
|:------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
15+
| base_uri | Full address of New Relic API end-point. By default the value points to the US end-point. If you want to use the EU end-point you can set this key to the following value:<br> - <a href="https://log-api.eu.newrelic.com/log/v1">https://log-api.eu.newrelic.com/log/v1</a> | <a href="https://log-api.newrelic.com/log/v1">https://log-api.newrelic.com/log/v1</a> |
16+
| api_key | Your key for data ingestion. The API key is also called the ingestion key, you can get more details on how to generated in the official documentation <a href="https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#event-insert-key">here</a>. From a configuration perspective either an `api_key` or a `license_key` is required. New Relic suggests to use primarily the `api_key`. | |
17+
| license_key | Optional authentication parameter for data ingestion. Note that New Relic suggest to use the `api_key` instead. You can read more about the License Key <a href="https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/license-key">here</a>. | |
18+
| compress | Set the compression mechanism for the payload. This option allows two values: `gzip` \(enabled by default\) or `false` to disable compression. | gzip |
19+
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | 0 |
20+
21+
The following configuration example, will emit a dummy example record and ingest it on New Relic. In your main configuration file append the following:
22+
23+
24+
{% tabs %}
25+
{% tab title="fluent-bit.yaml" %}
26+
27+
```yaml
28+
service:
29+
flush: 1
30+
log_level: info
31+
32+
pipeline:
33+
inputs:
34+
- name: dummy
35+
dummy: '{"message":"a simple message", "temp": "0.74", "extra": "false"}'
36+
samples: 1
37+
38+
outputs:
39+
- name: nrlogs
40+
match: '*'
41+
api_key: YOUR_API_KEY_HERE
42+
```
43+
44+
{% endtab %}
45+
{% tab title="fluent-bit.conf" %}
46+
47+
```text
8248
[SERVICE]
83-
flush 1
84-
log_level info
49+
flush 1
50+
log_level info
8551

8652
[INPUT]
87-
name dummy
88-
dummy {"message":"a simple message", "temp": "0.74", "extra": "false"}
89-
samples 1
53+
name dummy
54+
dummy {"message":"a simple message", "temp": "0.74", "extra": "false"}
55+
samples 1
9056

9157
[OUTPUT]
92-
name nrlogs
93-
match *
94-
api_key YOUR_API_KEY_HERE
58+
name nrlogs
59+
match *
60+
api_key YOUR_API_KEY_HERE
9561
```
9662

97-
run Fluent Bit with the new configuration file:
63+
{% endtab %}
64+
{% endtabs %}
9865

99-
```text
100-
fluent-bit -c newrelic.conf
66+
Run Fluent Bit with the new configuration file:
67+
68+
```shell
69+
# For YAML configuration.
70+
fluent-bit --config fluent-bit.yaml
71+
72+
# For classic configuration
73+
fluent-bit --config fluent-bit.conf
10174
```
10275

10376
Fluent Bit output:
10477

10578
```text
106-
Fluent Bit v1.5.0
107-
* Copyright (C) 2019-2020 The Fluent Bit Authors
108-
* Copyright (C) 2015-2018 Treasure Data
109-
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
110-
* https://fluentbit.io
111-
112-
[2020/04/10 10:58:32] [ info] [storage] version=1.0.3, initializing...
113-
[2020/04/10 10:58:32] [ info] [storage] in-memory
114-
[2020/04/10 10:58:32] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
115-
[2020/04/10 10:58:32] [ info] [engine] started (pid=2772591)
116-
[2020/04/10 10:58:32] [ info] [output:newrelic:newrelic.0] configured, hostname=log-api.newrelic.com:443
117-
[2020/04/10 10:58:32] [ info] [sp] stream processor started
79+
...
11880
[2020/04/10 10:58:35] [ info] [output:nrlogs:nrlogs.0] log-api.newrelic.com:443, HTTP status=202
11981
{"requestId":"feb312fe-004e-b000-0000-0171650764ac"}
120-
```
82+
...
83+
```

‎pipeline/outputs/null.md

Lines changed: 25 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,21 +14,39 @@ You can run the plugin from the command line or through the configuration file:
1414

1515
From the command line you can let Fluent Bit throws away events with the following options:
1616

17-
```bash
17+
```shell
1818
fluent-bit -i cpu -o null
1919
```
2020

2121
### Configuration File
2222

23-
In your main configuration file append the following Input & Output sections:
23+
In your main configuration file append the following:
2424

25-
```python
25+
{% tabs %}
26+
{% tab title="fluent-bit.yaml" %}
27+
28+
```yaml
29+
pipeline:
30+
inputs:
31+
- name: cpu
32+
tag: cpu
33+
34+
outputs:
35+
- name: null
36+
match: '*'
37+
```
38+
{% endtab %}
39+
{% tab title="fluent-bit.conf" %}
40+
41+
```text
2642
[INPUT]
27-
Name cpu
28-
Tag cpu
43+
Name cpu
44+
Tag cpu
2945

3046
[OUTPUT]
31-
Name null
32-
Match *
47+
Name null
48+
Match *
3349
```
3450

51+
{% endtab %}
52+
{% endtabs %}

‎pipeline/outputs/observe.md

Lines changed: 42 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2,28 +2,53 @@
22

33
Observe employs the **http** output plugin, allowing you to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html).
44

5-
For now the functionality is pretty basic and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
5+
For now the functionality is pretty basic, and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
66

77
The following are the specific HTTP parameters to employ:
88

99
## Configuration Parameters
1010

11-
| Key | Description | default |
12-
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
13-
| host | IP address or hostname of Observe's data collection endpoint. $(OBSERVE_CUSTOMER) is your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id) | OBSERVE_CUSTOMER.collect.observeinc.com |
14-
| port | TCP port of to employ when sending to Observe | 443 |
15-
| tls | Specify to use tls | on |
16-
| uri | Specify the HTTP URI for the Observe's data ingest | /v1/http/fluentbit |
17-
| format | The data format to be used in the HTTP request body | msgpack |
18-
| header | The specific header that provides the Observe token needed to authorize sending data [into a datastream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | Authorization Bearer ${OBSERVE_TOKEN} |
19-
| header | The specific header to instructs Observe how to decode incoming payloads | X-Observe-Decoder fluent |
20-
| compress | Set payload compression mechanism. Option available is 'gzip' | gzip |
21-
| tls.ca_file | **For use with Windows**: provide path to root cert | |
22-
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
11+
| Key | Description | default |
12+
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|
13+
| host | IP address or hostname of Observe's data collection endpoint. $(OBSERVE_CUSTOMER) is your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id) | OBSERVE_CUSTOMER.collect.observeinc.com |
14+
| port | TCP port of to employ when sending to Observe | 443 |
15+
| tls | Specify to use tls | on |
16+
| uri | Specify the HTTP URI for the Observe's data ingest | /v1/http/fluentbit |
17+
| format | The data format to be used in the HTTP request body | msgpack |
18+
| header | The specific header that provides the Observe token needed to authorize sending data [into a data stream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | Authorization Bearer ${OBSERVE_TOKEN} |
19+
| header | The specific header to instructs Observe how to decode incoming payloads | X-Observe-Decoder fluent |
20+
| compress | Set payload compression mechanism. Option available is 'gzip' | gzip |
21+
| tls.ca_file | **For use with Windows**: provide path to root cert | |
22+
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | 0 |
2323

2424
### Configuration File
2525

26-
In your main configuration file, append the following _Input_ & _Output_ sections:
26+
In your main configuration file append the following:
27+
28+
29+
{% tabs %}
30+
{% tab title="fluent-bit.yaml" %}
31+
32+
```yaml
33+
pipeline:
34+
35+
outputs:
36+
- name: http
37+
match: '*'
38+
host: my-observe-customer-id.collect.observeinc.com
39+
port: 443
40+
tls: on
41+
uri: /v1/http/fluentbit
42+
format: msgpack
43+
header:
44+
- 'Authorization Bearer ${OBSERVE_TOKEN}'
45+
- 'X-Observe-Decoder fluent'
46+
compress: gzip
47+
# For Windows: provide path to root cert
48+
#tls.ca_file C:\fluent-bit\isrgrootx1.pem
49+
```
50+
{% endtab %}
51+
{% tab title="fluent-bit.conf" %}
2752

2853
```text
2954
[OUTPUT]
@@ -42,5 +67,7 @@ In your main configuration file, append the following _Input_ & _Output_ section
4267
4368
# For Windows: provide path to root cert
4469
#tls.ca_file C:\fluent-bit\isrgrootx1.pem
45-
4670
```
71+
72+
{% endtab %}
73+
{% endtabs %}

0 commit comments

Comments
 (0)