Skip to content

Adjusted amazon outputs code examples to use 2 space indentations. Fixes #1890. #1906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 12, 2025
Merged

Adjusted amazon outputs code examples to use 2 space indentations. Fixes #1890. #1906

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
86a757d
Adjusted cloudwatch examples to use 2 space indents. Part of issue #1…
eschabell Jul 12, 2025
a8b7c72
Adjusted amazone kinesis data firehose examples to use 2 space indent…
eschabell Jul 12, 2025
6a42db5
Adjusted amazone kinesis data streams examples to use 2 space indents…
eschabell Jul 12, 2025
ea265ac
Adjusted amazone s3 examples to use 2 space indents. Part of issue #1…
eschabell Jul 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
266 changes: 133 additions & 133 deletions pipeline/outputs/cloudwatch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,26 +59,26 @@ In your main configuration file append the following:
```yaml
pipeline:

outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fluent-bit-cloudwatch
log_stream_prefix: from-fluent-bit-
auto_create_group: on
outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fluent-bit-cloudwatch
log_stream_prefix: from-fluent-bit-
auto_create_group: on
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group On
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group On
```

{% endtab %}
Expand All @@ -94,30 +94,30 @@ For an instance of `Localstack` running at `http://localhost:4566`, the followin
```yaml
pipeline:

outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fluent-bit-cloudwatch
log_stream_prefix: from-fluent-bit-
auto_create_group: on
endpoint: localhost
port: 4566
outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fluent-bit-cloudwatch
log_stream_prefix: from-fluent-bit-
auto_create_group: on
endpoint: localhost
port: 4566
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group On
endpoint localhost
port 4566
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fluent-bit-cloudwatch
log_stream_prefix from-fluent-bit-
auto_create_group On
endpoint localhost
port 4566
```

{% endtab %}
Expand All @@ -131,16 +131,16 @@ The following AWS IAM permissions are required to use this plugin:

```json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup",
"logs:PutLogEvents"
],
"Resource": "*"
}]
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup",
"logs:PutLogEvents"
],
"Resource": "*"
}]
}
```

Expand All @@ -154,20 +154,20 @@ Recall that the kubernetes filter can add metadata which will look like the foll

```text
kubernetes: {
annotations: {
"kubernetes.io/psp": "eks.privileged"
},
container_hash: "<some hash>",
container_name: "myapp",
docker_id: "<some id>",
host: "ip-10-1-128-166.us-east-2.compute.internal",
labels: {
app: "myapp",
"pod-template-hash": "<some hash>"
},
namespace_name: "my-namespace",
pod_id: "198f7dd2-2270-11ea-be47-0a5d932f5920",
pod_name: "myapp-5468c5d4d7-n2swr"
annotations: {
"kubernetes.io/psp": "eks.privileged"
},
container_hash: "<some hash>",
container_name: "myapp",
docker_id: "<some id>",
host: "ip-10-1-128-166.us-east-2.compute.internal",
labels: {
app: "myapp",
"pod-template-hash": "<some hash>"
},
namespace_name: "my-namespace",
pod_id: "198f7dd2-2270-11ea-be47-0a5d932f5920",
pod_name: "myapp-5468c5d4d7-n2swr"
}
```

Expand All @@ -181,30 +181,30 @@ Here is our configuration:
```yaml
pipeline:

outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fallback-group
log_stream_prefix: fallback-stream
auto_create_group: on
log_group_template: application-logs-$kubernetes['host'].$kubernetes['namespace_name']
log_stream_template: $kubernetes['pod_name'].$kubernetes['container_name']
outputs:
- name: cloudwatch_logs
match: '*'
region: us-east-1
log_group_name: fallback-group
log_stream_prefix: fallback-stream
auto_create_group: on
log_group_template: application-logs-$kubernetes['host'].$kubernetes['namespace_name']
log_stream_template: $kubernetes['pod_name'].$kubernetes['container_name']
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fallback-group
log_stream_prefix fallback-stream
auto_create_group On
log_group_template application-logs-$kubernetes['host'].$kubernetes['namespace_name']
log_stream_template $kubernetes['pod_name'].$kubernetes['container_name']
Name cloudwatch_logs
Match *
region us-east-1
log_group_name fallback-group
log_stream_prefix fallback-stream
auto_create_group On
log_group_template application-logs-$kubernetes['host'].$kubernetes['namespace_name']
log_stream_template $kubernetes['pod_name'].$kubernetes['container_name']
```

{% endtab %}
Expand Down Expand Up @@ -254,54 +254,54 @@ For using the `mem` input plugin and sending memory usage metrics to CloudWatch,

```yaml
service:
log_level: info
log_level: info

pipeline:
inputs:
- name: mem
tag: mem
inputs:
- name: mem
tag: mem

filters:
- name: aws
match: '*'
filters:
- name: aws
match: '*'

outputs:
- name: cloudwatch_logs
match: '*'
region: us-west-2
log_stream_name: fluent-bit-cloudwatch
log_group_name: fluent-bit-cloudwatch
log_format: json/emf
metric_namespace: fluent-bit-metrics
metric_dimensions: ec2_instance_id
auto_create_group: true
outputs:
- name: cloudwatch_logs
match: '*'
region: us-west-2
log_stream_name: fluent-bit-cloudwatch
log_group_name: fluent-bit-cloudwatch
log_format: json/emf
metric_namespace: fluent-bit-metrics
metric_dimensions: ec2_instance_id
auto_create_group: true
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[SERVICE]
Log_Level info
Log_Level info

[INPUT]
Name mem
Tag mem
Name mem
Tag mem

[FILTER]
Name aws
Match *
Name aws
Match *

[OUTPUT]
Name cloudwatch_logs
Match *
region us-west-2
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
log_format json/emf
metric_namespace fluent-bit-metrics
metric_dimensions ec2_instance_id
auto_create_group true
Name cloudwatch_logs
Match *
region us-west-2
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
log_format json/emf
metric_namespace fluent-bit-metrics
metric_dimensions ec2_instance_id
auto_create_group true
```

{% endtab %}
Expand All @@ -314,54 +314,54 @@ The following config will set two dimensions to all of our metrics- `ec2_instanc

```yaml
service:
log_level: info
log_level: info

pipeline:
inputs:
- name: mem
tag: mem
inputs:
- name: mem
tag: mem

filters:
- name: aws
match: '*'
filters:
- name: aws
match: '*'

outputs:
- name: cloudwatch_logs
match: '*'
region: us-west-2
log_stream_name: fluent-bit-cloudwatch
log_group_name: fluent-bit-cloudwatch
log_format: json/emf
metric_namespace: fluent-bit-metrics
metric_dimensions: ec2_instance_id,az
auto_create_group: true
outputs:
- name: cloudwatch_logs
match: '*'
region: us-west-2
log_stream_name: fluent-bit-cloudwatch
log_group_name: fluent-bit-cloudwatch
log_format: json/emf
metric_namespace: fluent-bit-metrics
metric_dimensions: ec2_instance_id,az
auto_create_group: true
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[SERVICE]
Log_Level info
Log_Level info

[INPUT]
Name mem
Tag mem
Name mem
Tag mem

[FILTER]
Name aws
Match *
Name aws
Match *

[OUTPUT]
Name cloudwatch_logs
Match *
region us-west-2
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
log_format json/emf
metric_namespace fluent-bit-metrics
metric_dimensions ec2_instance_id,az
auto_create_group true
Name cloudwatch_logs
Match *
region us-west-2
log_stream_name fluent-bit-cloudwatch
log_group_name fluent-bit-cloudwatch
log_format json/emf
metric_namespace fluent-bit-metrics
metric_dimensions ec2_instance_id,az
auto_create_group true
```

{% endtab %}
Expand Down
Loading