You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/outputs/loki.md
+36-10Lines changed: 36 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,18 @@
1
1
# Loki
2
2
3
-
[Loki](https://grafana.com/oss/loki/) is multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate.
3
+
[Loki](https://grafana.com/oss/loki/) is multi-tenant log aggregation system inspired by Prometheus.
4
+
It is designed to be very cost effective and easy to operate.
4
5
5
-
The Fluent Bit `loki` built-in output plugin allows you to send your log or events to a Loki service. It support data enrichment with Kubernetes labels, custom label keys and Tenant ID within others.
6
+
The Fluent Bit `loki` built-in output plugin allows you to send your log or events to a Loki service.
7
+
It supports data enrichment with Kubernetes labels, custom label keys and Tenant ID within others.
8
+
9
+
Be aware there is a separate Golang output plugin provided by [Grafana](https://grafana.com/docs/loki/latest/clients/fluentbit/) with different configuration options.
6
10
7
11
## Configuration Parameters
8
12
9
13
| Key | Description | Default |
10
14
| :--- | :--- | :--- |
11
-
| host | Loki hostname or IP address | 127.0.0.1 |
15
+
| host | Loki hostname or IP address. Do not include the subpath, i.e. `loki/api/v1/push`, but just the base hostname/URL. | 127.0.0.1 |
12
16
| port | Loki TCP port | 3100 |
13
17
| http\_user | Set HTTP basic authentication user name ||
14
18
| http\_passwd | Set HTTP basic authentication password ||
@@ -25,7 +29,8 @@ The Fluent Bit `loki` built-in output plugin allows you to send your log or even
25
29
26
30
Loki store the record logs inside Streams, a stream is defined by a set of labels, at least one label is required.
27
31
28
-
Fluent Bit implements a flexible mechanism to set labels by using fixed key/value pairs of text but also allowing to set as labels certain keys that exists as part of the records that are being processed. Consider the following JSON record \(pretty printed for readability\):
32
+
Fluent Bit implements a flexible mechanism to set labels by using fixed key/value pairs of text but also allowing to set as labels certain keys that exists as part of the records that are being processed.
33
+
Consider the following JSON record \(pretty printed for readability\):
29
34
30
35
```javascript
31
36
{
@@ -51,7 +56,8 @@ If you decide that your Loki Stream will be composed by two labels called `job`
51
56
labels job=fluentbit, $sub['stream']
52
57
```
53
58
54
-
As you can see the label `job` has the value `fluentbit` and the second label is configured to access the nested map called `sub` targeting the value of the key `stream` . Note that the second label name **must** starts with a `$`, that means that's a [Record Accessor](../../administration/configuring-fluent-bit/record-accessor.md) pattern so it provide you the ability to retrieve values from nested maps by using the key names.
59
+
As you can see the label `job` has the value `fluentbit` and the second label is configured to access the nested map called `sub` targeting the value of the key `stream` .
60
+
Note that the second label name **must** starts with a `$`, that means that's a [Record Accessor](../../administration/configuring-fluent-bit/record-accessor.md) pattern so it provide you the ability to retrieve values from nested maps by using the key names.
55
61
56
62
When processing above's configuration, internally the ending labels for the stream in question becomes:
Another feature of Labels management is the ability to provide custom key names, using the same record accessor pattern we can specify the key name manually and let the value to be populated automatically at runtime, e.g:
The additional configuration property called `label_keys` allow to specify multiple record keys that needs to be placed as part of the outgoing Stream Labels, yes, this is a similar feature than the one explained above in the `labels` property. Consider this as another way to set a record key in the Stream, but with the limitation that you cannot use a custom name for the key value.
85
+
The additional configuration property called `label_keys` allow to specify multiple record keys that needs to be placed as part of the outgoing Stream Labels, yes, this is a similar feature than the one explained above in the `labels` property.
86
+
Consider this as another way to set a record key in the Stream, but with the limitation that you cannot use a custom name for the key value.
80
87
81
88
The following configuration examples generate the same Stream Labels:
82
89
@@ -130,11 +137,30 @@ This plugin inherit core Fluent Bit features to customize the network behavior a
130
137
131
138
Note that all options mentioned in the articles above must be enabled in the plugin configuration in question.
132
139
140
+
### Fluent Bit + Grafana Cloud
141
+
142
+
Fluent Bit supports sending logs (and metrics) to [Grafana Cloud](https://grafana.com/products/cloud/) by providing the appropriate URL and ensuring TLS is enabled.
143
+
144
+
An example configuration - make sure to set the credentials and ensure the host URL matches the correct one for your deployment:
145
+
146
+
```text
147
+
[OUTPUT]
148
+
Name loki
149
+
Match *
150
+
Host logs-prod-eu-west-0.grafana.net
151
+
port 443
152
+
tls on
153
+
tls.verify on
154
+
http_user XXX
155
+
http_passwd XXX
156
+
```
157
+
133
158
## Getting Started
134
159
135
-
The following configuration example, will emit a dummy example record and ingest it on Loki . Copy and paste the following content in a file called `out_loki.conf`:
160
+
The following configuration example, will emit a dummy example record and ingest it on Loki .
161
+
Copy and paste the following content into a file called `out_loki.conf`:
136
162
137
-
```python
163
+
```text
138
164
[SERVICE]
139
165
flush 1
140
166
log_level info
@@ -149,7 +175,7 @@ The following configuration example, will emit a dummy example record and ingest
0 commit comments