Skip to content

Commit 95ce2ea

Browse files
committed
address review feedback
Signed-off-by: braydonk <[email protected]>
1 parent 5a586ce commit 95ce2ea

File tree

2 files changed

+20
-9
lines changed

2 files changed

+20
-9
lines changed

pipeline/outputs/stackdriver.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,15 @@ For instance, for a K8s resource type, `resource_labels` can be used in tandem w
157157
```
158158

159159
`resource_labels` also supports validation for required labels based on the input resource type. This allows fluent-bit to check if all specified labels are present for a given configuration before runtime. If validation is not currently supported for a resource type that you would like to use this API with, we encourage you to open a pull request for it. Adding validation for a new resource type is simple - all that is needed is to specify the resources associated with the type alongside the required labels [here](https://github.com/fluent/fluent-bit/blob/master/plugins/out_stackdriver/stackdriver_resource_types.c#L27).
160+
161+
## Log Name
162+
163+
By default, the plugin will write to the following log name:
164+
```
165+
/projects/<project ID>/logs/<log tag>
166+
```
167+
You may be in a scenario where being more specific about the log name is important (for example [integration with Log Router rules](https://cloud.google.com/logging/docs/routing/overview) or [controlling cardinality of log based metrics]((https://cloud.google.com/logging/docs/logs-based-metrics/troubleshooting#too-many-time-series))). You can control the log name directly on a per-log basis by using the [`logging.googleapis.com/logName` special field][StackdriverSpecialFields]. You can configure a `log_name_key` if you'd like to use something different than `logging.googleapis.com/logName`, i.e. if the `log_name_key` is set to `mylognamefield` will extract the log name from `mylognamefield` in the log.
168+
160169
## Troubleshooting Notes
161170

162171
### Upstream connection error

pipeline/outputs/stackdriver_special_fields.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ Currently, we support some special fields in fluent-bit for setting fields on th
2626
| JSON log field | Description |
2727
| :--- | :--- |
2828
| `logging.googleapis.com/projectId` | Changes the project ID that this log will be written to. Ensure that you are authenticated to write logs to this project. |
29-
| `logging.googleapis.com/local_resource_id` | Overrides the [configured `local_resource_id`](). |
29+
| `logging.googleapis.com/local_resource_id` | Overrides the [configured `local_resource_id`](./stackdriver.md#resource-labels). |
3030

3131
## Using Special Fields
3232

@@ -61,7 +61,7 @@ Adding special fields to logs is best done through the [`modify` filter](https:/
6161

6262
For special fields with simple types (with the exception of the [`logging.googleapis.com/insertId` field](#insert-id)), they will follow this pattern (demonstrated with the `logging.googleapis.com/logName` field):
6363

64-
**1. If the special field matches the type, it will be moved to the corresponding LogEntry field.** For example:
64+
1. If the special field matches the type, it will be moved to the corresponding LogEntry field. For example:
6565
```text
6666
{
6767
"logging.googleapis.com/logName": "my_log"
@@ -79,7 +79,7 @@ the logEntry will be:
7979
}
8080
```
8181

82-
**2. If the field is non-empty but an invalid, it will be left in the jsonPayload.** For example:
82+
2. If the field is non-empty but an invalid, it will be left in the jsonPayload. For example:
8383
```text
8484
{
8585
"logging.googleapis.com/logName": 12345
@@ -132,7 +132,7 @@ The `timestampSecond` and `timestampNano` fields don't map directly to the `time
132132

133133
For special fields that expect the format of a proto type from the `LogEntry` (with the exception of the `logging.googleapis.com/monitored_resource` field) will follow this pattern (demonstrated with the `logging.googleapis.com/operation` field):
134134

135-
**If any subfields of the proto are empty or in incorrect type, the plugin will set these subfields empty.** For example:
135+
If any subfields of the proto are empty or in incorrect type, the plugin will set these subfields empty. For example:
136136
```text
137137
{
138138
"logging.googleapis.com/operation": {
@@ -158,7 +158,7 @@ the logEntry will be:
158158
}
159159
```
160160

161-
**If the field itself is not a map, the plugin will leave this field untouched.** For example:
161+
If the field itself is not a map, the plugin will leave this field untouched. For example:
162162
```text
163163
{
164164
"logging.googleapis.com/operation": "some string",
@@ -176,7 +176,7 @@ the logEntry will be:
176176
}
177177
```
178178

179-
**If there are extra subfields, the plugin will add the recognized fields to the corresponding field in the LogEntry, and preserve the extra subfields in jsonPayload.** For example:
179+
If there are extra subfields, the plugin will add the recognized fields to the corresponding field in the LogEntry, and preserve the extra subfields in jsonPayload. For example:
180180
```text
181181
{
182182
"logging.googleapis.com/operation": {
@@ -221,7 +221,7 @@ The `logging.googleapis.com/monitored_resource` field is parsed in a special way
221221

222222
The `type` field from the [`MonitoredResource` proto]() is not parsed out of the special field. It is read from the [`resource` plugin configuration option](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver#configuration-parameters). If it is supplied in the `logging.googleapis.com/monitored_resource` special field, it will not be recognized.
223223

224-
The `labels` field is expected to be an `object<string, string>`. If any fields have a value that is not a string, the value will be ignored and **not preserved**. The plugin will log an error and drop the field.
224+
The `labels` field is expected to be an `object<string, string>`. If any fields have a value that is not a string, the value will be ignored and not preserved. The plugin will log an error and drop the field.
225225

226226
If no valid `labels` field is found, or if all of entries in the `labels` object provided are invalid, the `logging.googleapis.com/monitored_resource` field is dropped in favour of automatically setting resource labels using other available information based on the configured `resource` type.
227227

@@ -253,7 +253,8 @@ If one of the following JSON timestamp representations is present in a structure
253253

254254
Without time-related fields, the plugin will set the current time as timestamp.
255255

256-
**Format 1**
256+
### Format 1
257+
257258
Set the input log as followed:
258259
```text
259260
{
@@ -275,7 +276,8 @@ the logEntry will be:
275276
}
276277
```
277278

278-
**Format 2**
279+
### Format 2
280+
279281
Set the input log as followed:
280282
```text
281283
{

0 commit comments

Comments
 (0)