Skip to content

Adding YAML examples, fixing standards for table layouts for output plugins form range N and O. Fixes #1926. #1930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 17, 2025
Merged

Adding YAML examples, fixing standards for table layouts for output plugins form range N and O. Fixes #1926. #1930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f78793b
Adding YAML examples and fixing table layout. Part of isssue #1926.
eschabell Jul 16, 2025
f6e458b
Adding YAML examples to NULL output plugin doc. Part of isssue #1926.
eschabell Jul 16, 2025
8da623f
Adding YAML examples and fixing table layout for Observe output plugi…
eschabell Jul 16, 2025
b73b551
Adding back missing original OpenObserve output plugin doc, adding YA…
eschabell Jul 16, 2025
d4bd2aa
Adding YAML examples and fixing table layouts for OpenSearch output p…
eschabell Jul 16, 2025
2a9ec0a
Adding YAML examples and fixing table layouts for OpenTelemetry outpu…
eschabell Jul 16, 2025
f8f925a
Adding YAML examples and fixing table layouts for Oracle Log Analytic…
eschabell Jul 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 56 additions & 93 deletions pipeline/outputs/new-relic.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,111 +10,74 @@ Before to get started with the plugin configuration, make sure to obtain the pro

## Configuration Parameters

| Key | Description | Default |
| :--- | :--- | :--- |


<table>
<thead>
<tr>
<th style="text-align:left">base_uri</th>
<th style="text-align:left">
<p>Full address of New Relic API end-point. By default the value points to
the US end-point.
<br />
</p>
<p>If you want to use the EU end-point you can set this key to the following
value:
<br />
<br /><a href="https://log-api.eu.newrelic.com/log/v1">https://log-api.eu.newrelic.com/log/v1</a>
</p>
</th>
<th style="text-align:left"><a href="https://log-api.newrelic.com/log/v1">https://log-api.newrelic.com/log/v1</a>
</th>
</tr>
</thead>
<tbody></tbody>
</table>

<table>
<thead>
<tr>
<th style="text-align:left">api_key</th>
<th style="text-align:left">
<p>Your key for data ingestion. The API key is also called the ingestion
key, you can get more details on how to generated in the official documentation
<a
href="https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#event-insert-key">here</a>.</p>
<p>From a configuration perspective either an <code>api_key</code> or an <code>license_key</code> is
required. New Relic suggest to use primary the <code>api_key</code>.</p>
</th>
<th style="text-align:left"></th>
</tr>
</thead>
<tbody></tbody>
</table>

<table>
<thead>
<tr>
<th style="text-align:left">license_key</th>
<th style="text-align:left">
<p>Optional authentication parameter for data ingestion.
<br />
</p>
<p>Note that New Relic suggest to use the <code>api_key</code> instead. You
can read more about the License Key <a href="https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/license-key">here</a>.</p>
</th>
<th style="text-align:left"></th>
</tr>
</thead>
<tbody></tbody>
</table>

| compress | Set the compression mechanism for the payload. This option allows two values: `gzip` \(enabled by default\) or `false` to disable compression. | gzip |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
| :--- | :--- | :--- |


The following configuration example, will emit a dummy example record and ingest it on New Relic. Copy and paste the following content in a file called `newrelic.conf`:

```python
| Key | Description | Default |
|:------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
| base_uri | Full address of New Relic API end-point. By default the value points to the US end-point. If you want to use the EU end-point you can set this key to the following value:<br> - <a href="https://log-api.eu.newrelic.com/log/v1">https://log-api.eu.newrelic.com/log/v1</a> | <a href="https://log-api.newrelic.com/log/v1">https://log-api.newrelic.com/log/v1</a> |
| api_key | Your key for data ingestion. The API key is also called the ingestion key, you can get more details on how to generated in the official documentation <a href="https://docs.newrelic.com/docs/apis/get-started/intro-apis/types-new-relic-api-keys#event-insert-key">here</a>. From a configuration perspective either an `api_key` or a `license_key` is required. New Relic suggests to use primarily the `api_key`. | |
| license_key | Optional authentication parameter for data ingestion. Note that New Relic suggest to use the `api_key` instead. You can read more about the License Key <a href="https://docs.newrelic.com/docs/accounts/install-new-relic/account-setup/license-key">here</a>. | |
| compress | Set the compression mechanism for the payload. This option allows two values: `gzip` \(enabled by default\) or `false` to disable compression. | gzip |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | 0 |

The following configuration example, will emit a dummy example record and ingest it on New Relic. In your main configuration file append the following:


{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
service:
flush: 1
log_level: info

pipeline:
inputs:
- name: dummy
dummy: '{"message":"a simple message", "temp": "0.74", "extra": "false"}'
samples: 1

outputs:
- name: nrlogs
match: '*'
api_key: YOUR_API_KEY_HERE
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[SERVICE]
flush 1
log_level info
flush 1
log_level info

[INPUT]
name dummy
dummy {"message":"a simple message", "temp": "0.74", "extra": "false"}
samples 1
name dummy
dummy {"message":"a simple message", "temp": "0.74", "extra": "false"}
samples 1

[OUTPUT]
name nrlogs
match *
api_key YOUR_API_KEY_HERE
name nrlogs
match *
api_key YOUR_API_KEY_HERE
```

run Fluent Bit with the new configuration file:
{% endtab %}
{% endtabs %}

```text
fluent-bit -c newrelic.conf
Run Fluent Bit with the new configuration file:

```shell
# For YAML configuration.
fluent-bit --config fluent-bit.yaml

# For classic configuration
fluent-bit --config fluent-bit.conf
```

Fluent Bit output:

```text
Fluent Bit v1.5.0
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2020/04/10 10:58:32] [ info] [storage] version=1.0.3, initializing...
[2020/04/10 10:58:32] [ info] [storage] in-memory
[2020/04/10 10:58:32] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2020/04/10 10:58:32] [ info] [engine] started (pid=2772591)
[2020/04/10 10:58:32] [ info] [output:newrelic:newrelic.0] configured, hostname=log-api.newrelic.com:443
[2020/04/10 10:58:32] [ info] [sp] stream processor started
...
[2020/04/10 10:58:35] [ info] [output:nrlogs:nrlogs.0] log-api.newrelic.com:443, HTTP status=202
{"requestId":"feb312fe-004e-b000-0000-0171650764ac"}
```
...
```
32 changes: 25 additions & 7 deletions pipeline/outputs/null.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,39 @@ You can run the plugin from the command line or through the configuration file:

From the command line you can let Fluent Bit throws away events with the following options:

```bash
```shell
fluent-bit -i cpu -o null
```

### Configuration File

In your main configuration file append the following Input & Output sections:
In your main configuration file append the following:

```python
{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
- name: cpu
tag: cpu

outputs:
- name: null
match: '*'
```
{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[INPUT]
Name cpu
Tag cpu
Name cpu
Tag cpu

[OUTPUT]
Name null
Match *
Name null
Match *
```

{% endtab %}
{% endtabs %}
57 changes: 42 additions & 15 deletions pipeline/outputs/observe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,53 @@

Observe employs the **http** output plugin, allowing you to flush your records [into Observe](https://docs.observeinc.com/en/latest/content/data-ingestion/forwarders/fluentbit.html).

For now the functionality is pretty basic and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.
For now the functionality is pretty basic, and it issues a POST request with the data records in [MessagePack](http://msgpack.org) (or JSON) format.

The following are the specific HTTP parameters to employ:

## Configuration Parameters

| Key | Description | default |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
| host | IP address or hostname of Observe's data collection endpoint. $(OBSERVE_CUSTOMER) is your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id) | OBSERVE_CUSTOMER.collect.observeinc.com |
| port | TCP port of to employ when sending to Observe | 443 |
| tls | Specify to use tls | on |
| uri | Specify the HTTP URI for the Observe's data ingest | /v1/http/fluentbit |
| format | The data format to be used in the HTTP request body | msgpack |
| header | The specific header that provides the Observe token needed to authorize sending data [into a datastream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | Authorization Bearer ${OBSERVE_TOKEN} |
| header | The specific header to instructs Observe how to decode incoming payloads | X-Observe-Decoder fluent |
| compress | Set payload compression mechanism. Option available is 'gzip' | gzip |
| tls.ca_file | **For use with Windows**: provide path to root cert | |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
| Key | Description | default |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|
| host | IP address or hostname of Observe's data collection endpoint. $(OBSERVE_CUSTOMER) is your [Customer ID](https://docs.observeinc.com/en/latest/content/common-topics/HelpfulHints.html?highlight=customer%20id#customer-id) | OBSERVE_CUSTOMER.collect.observeinc.com |
| port | TCP port of to employ when sending to Observe | 443 |
| tls | Specify to use tls | on |
| uri | Specify the HTTP URI for the Observe's data ingest | /v1/http/fluentbit |
| format | The data format to be used in the HTTP request body | msgpack |
| header | The specific header that provides the Observe token needed to authorize sending data [into a data stream](https://docs.observeinc.com/en/latest/content/data-ingestion/datastreams.html?highlight=ingest%20token#create-a-datastream). | Authorization Bearer ${OBSERVE_TOKEN} |
| header | The specific header to instructs Observe how to decode incoming payloads | X-Observe-Decoder fluent |
| compress | Set payload compression mechanism. Option available is 'gzip' | gzip |
| tls.ca_file | **For use with Windows**: provide path to root cert | |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | 0 |

### Configuration File

In your main configuration file, append the following _Input_ & _Output_ sections:
In your main configuration file append the following:


{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:

outputs:
- name: http
match: '*'
host: my-observe-customer-id.collect.observeinc.com
port: 443
tls: on
uri: /v1/http/fluentbit
format: msgpack
header:
- 'Authorization Bearer ${OBSERVE_TOKEN}'
- 'X-Observe-Decoder fluent'
compress: gzip
# For Windows: provide path to root cert
#tls.ca_file C:\fluent-bit\isrgrootx1.pem
```
{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Expand All @@ -42,5 +67,7 @@ In your main configuration file, append the following _Input_ & _Output_ section

# For Windows: provide path to root cert
#tls.ca_file C:\fluent-bit\isrgrootx1.pem

```

{% endtab %}
{% endtabs %}
Loading