Skip to content

Commit 5a752e4

Browse files
committed
Adding YAML examples and cleanup to Amazon Kinesis Data Streams output plugin. Part of issue #1890.
Signed-off-by: Eric D. Schabell <[email protected]>
1 parent 1d69633 commit 5a752e4

File tree

1 file changed

+27
-10
lines changed

1 file changed

+27
-10
lines changed

pipeline/outputs/kinesis.md

Lines changed: 27 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,6 @@ description: Send logs to Amazon Kinesis Streams
44

55
# Amazon Kinesis Data Streams
66

7-
![](../../.gitbook/assets/image%20%288%29.png)
8-
97
The Amazon Kinesis Data Streams output plugin allows to ingest your records into the [Kinesis](https://aws.amazon.com/kinesis/data-streams/) service.
108

119
This is the documentation for the core Fluent Bit Kinesis plugin written in C. It has all the core features of the [aws/amazon-kinesis-streams-for-fluent-bit](https://github.com/aws/amazon-kinesis-streams-for-fluent-bit) Golang Fluent Bit plugin released in 2019. The Golang plugin was named `kinesis`; this new high performance and highly efficient kinesis plugin is called `kinesis_streams` to prevent conflicts/confusion.
@@ -40,13 +38,29 @@ In order to send records into Amazon Kinesis Data Streams, you can run the plugi
4038

4139
The **kinesis\_streams** plugin, can read the parameters from the command line through the **-p** argument \(property\), e.g:
4240

43-
```text
41+
```shell
4442
fluent-bit -i cpu -o kinesis_streams -p stream=my-stream -p region=us-west-2 -m '*' -f 1
4543
```
4644

4745
### Configuration File
4846

49-
In your main configuration file append the following _Output_ section:
47+
In your main configuration file append the following:
48+
49+
{% tabs %}
50+
{% tab title="fluent-bit.yaml" %}
51+
52+
```yaml
53+
pipeline:
54+
55+
outputs:
56+
- name: kinesis_steams
57+
match: '*'
58+
region: us-east-1
59+
stream: my-stream
60+
```
61+
62+
{% endtab %}
63+
{% tab title="fluent-bit.conf" %}
5064
5165
```text
5266
[OUTPUT]
@@ -56,11 +70,14 @@ In your main configuration file append the following _Output_ section:
5670
stream my-stream
5771
```
5872

73+
{% endtab %}
74+
{% endtabs %}
75+
5976
### Permissions
6077

6178
The following AWS IAM permissions are required to use this plugin:
6279

63-
```
80+
```json
6481
{
6582
"Version": "2012-10-17",
6683
"Statement": [{
@@ -87,19 +104,19 @@ Amazon distributes a container image with Fluent Bit and these plugins.
87104

88105
Our images are available in Amazon ECR Public Gallery. You can download images with different tags by following command:
89106

90-
```text
107+
```shell
91108
docker pull public.ecr.aws/aws-observability/aws-for-fluent-bit:<tag>
92109
```
93110

94111
For example, you can pull the image with latest version by:
95112

96-
```text
113+
```shell
97114
docker pull public.ecr.aws/aws-observability/aws-for-fluent-bit:latest
98115
```
99116

100117
If you see errors for image pull limits, try log into public ECR with your AWS credentials:
101118

102-
```text
119+
```shell
103120
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
104121
```
105122

@@ -113,8 +130,8 @@ You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/A
113130

114131
You can use our SSM Public Parameters to find the Amazon ECR image URI in your region:
115132

116-
```text
133+
```shell
117134
aws ssm get-parameters-by-path --path /aws/service/aws-for-fluent-bit/
118135
```
119136

120-
For more see [the AWS for Fluent Bit github repo](https://github.com/aws/aws-for-fluent-bit#public-images).
137+
For more see [the AWS for Fluent Bit github repo](https://github.com/aws/aws-for-fluent-bit#public-images).

0 commit comments

Comments
 (0)