You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pipeline/outputs/oci-logging-analytics.md
+40-49Lines changed: 40 additions & 49 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,56 +4,47 @@ description: Send logs to Oracle Cloud Infrastructure Logging Analytics Service
4
4
5
5
# Oracle Cloud Infrastructure Logging Analytics
6
6
7
-
Oracle Cloud Infrastructure Logging Analytics output plugin allows you to ingest your log records into [OCI Logging Analytics](https://www.oracle.com/manageability/logging-analytics/) service.
7
+
The _Oracle Cloud Infrastructure Logging Analytics_ output plugin lets you ingest your log records into the [Oracle Cloud Infrastructure (OCI) Logging Analytics](https://docs.oracle.com/en-us/iaas/logging-analytics/home.htm) service.
8
8
9
-
Oracle Cloud Infrastructure Logging Analytics is a machine learning-based cloud service that monitors, aggregates, indexes, and analyzes all log data from on-premises and multicloud environments. Enabling users to search, explore, and correlate this data to troubleshoot and resolve problems faster and derive insights to make better operational decisions.
9
+
## Configuration parameters
10
10
11
-
For details about OCI Logging Analytics refer to [the documentation](https://docs.oracle.com/en-us/iaas/logging-analytics/index.html).
11
+
This plugin uses the following configuration parameters:
12
12
13
-
## Configuration Parameters
13
+
| Key | Description | Default |
14
+
| --- | ----------- | ------- |
15
+
|`config_file_location`| The location of the [configuration file](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm#SDK_and_CLI_Configuration_File) that contains OCI authentication details. |_none_|
16
+
|`profile_name`| The OCI configuration profile name to be used from the configuration file. |`DEFAULT`|
17
+
|`namespace`| The OCI tenancy namespace to upload log data to. |_none_|
18
+
|`proxy`| The proxy name, in `http://host:port` format. Only supports HTTP protocol. |_none_|
19
+
|`workers`| The number of [workers](../administration/multithreading.md#outputs) to perform flush operations for this output. |`1`|
20
+
|`oci_config_in_record`| If set to `true`, the following `oci_la_*` will be read from the record itself instead of the output plugin configuration. |`false`|
21
+
|`oci_la_log_group_id`| Required. The Oracle Cloud Identifier (OCID) of the Logging Analytics where you want to store logs. |_none_|
22
+
|`oci_la_log_source_name`| Required. The Logging Analytics Source to use for processing log records. |_none_|
23
+
|`oci_la_entity_id`| The OCID of the Logging Analytics entity. |_none_|
24
+
|`oci_la_entity_type`| The entity type of the Logging Analytics entity. |_none_|
25
+
|`oci_la_log_path`| Specifies the original location of the log files. |_none_|
26
+
|`oci_la_global_metadata`| Specifies additional global metadata along with original log content to Logging Analytics. The format is `key_name value`. This option can be set multiple times. |_none_|
27
+
|`oci_la_metadata`| Specifies additional metadata for a log event along with original log content to Logging Analytics. The format is `key_name value`. This option can be set multiple times. |_none_|
14
28
15
-
Following are the top level configuration properties of the plugin:
| config_file_location | The location of the configuration file containing OCI authentication details. Reference for generating the configuration file - https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm#SDK_and_CLI_Configuration_File||
20
-
| profile_name | OCI Config Profile Name to be used from the configuration file | DEFAULT |
21
-
| namespace | OCI Tenancy Namespace in which the collected log data is to be uploaded ||
22
-
| proxy | define proxy if required, in http://host:port format, supports only http protocol ||
23
-
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. |`1`|
31
+
The OCI Logging Analytics output plugin supports TLS/SSL. For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).
24
32
25
-
The following parameters are to set the Logging Analytics resources that must be used to process your logs by OCI Logging Analytics.
| oci_config_in_record | If set to true, the following oci_la_* will be read from the record itself instead of the output plugin configuration. | false |
30
-
| oci_la_log_group_id | The OCID of the Logging Analytics Log Group where the logs must be stored. This is a mandatory parameter ||
31
-
| oci_la_log_source_name | The Logging Analytics Source that must be used to process the log records. This is a mandatory parameter ||
32
-
| oci_la_entity_id | The OCID of the Logging Analytics Entity ||
33
-
| oci_la_entity_type | The entity type of the Logging Analytics Entity ||
34
-
| oci_la_log_path | Specify the original location of the log files ||
35
-
| oci_la_global_metadata | Use this parameter to specify additional global metadata along with original log content to Logging Analytics. The format is 'key_name value'. This option can be set multiple times ||
36
-
| oci_la_metadata | Use this parameter to specify additional metadata for a log event along with original log content to Logging Analytics. The format is 'key_name value'. This option can be set multiple times ||
37
-
38
-
## TLS/SSL
39
-
40
-
The OCI Logging Analytics output plugin supports TLS/SSL.
41
-
For more details about the properties available and general configuration, see [TLS/SSL](../../administration/transport-security.md).
42
-
43
-
## Getting Started
33
+
## Get started
44
34
45
35
### Prerequisites
46
36
47
-
- OCI Logging Analytics service must be onboarded with the minimum required policies, in the OCI region where you want to monitor. Refer [Logging Analytics Quick Start](https://docs.oracle.com/en-us/iaas/logging-analytics/doc/quick-start.html) for details.
48
-
- Create OCI Logging Analytics LogGroup(s) if not done already. Refer [Create Log Group](https://docs.oracle.com/en-us/iaas/logging-analytics/doc/create-logging-analytics-resources.html#GUID-D1758CFB-861F-420D-B12F-34D1CC5E3E0E) for details.
37
+
- You must onboard with the OCI Logging Analytics service for the minimum required policies in the OCI region where you want to monitor. Refer to [Logging Analytics Quick Start](https://docs.oracle.com/en-us/iaas/logging-analytics/doc/quick-start.html) for details.
38
+
39
+
- You must create one or more OCI Logging Analytics log groups. Refer to [Create Log Group](https://docs.oracle.com/en-us/iaas/logging-analytics/doc/create-logging-analytics-resources.html#GUID-D1758CFB-861F-420D-B12F-34D1CC5E3E0E) for details.
49
40
50
-
### Running the output plugin
41
+
### Run the output plugin
51
42
52
-
In order to insert records into the OCI Logging Analytics service, you can run the plugin from the command line or through the configuration file:
43
+
To insert records into the OCI Logging Analytics service, you can run the plugin from the command line or through a configuration file.
53
44
54
45
#### Command line
55
46
56
-
The OCI Logging Analytics plugin can read the parameters from the command line in two ways, through the -p argument (property), e.g:
47
+
The OCI Logging Analytics plugin can read the parameters from the command line in two ways, through the `-p`(property) argument. For example:
In case of multiple inputs, where `oci_la_*` properties can differ, you can add the properties in the record itself and instruct the plugin to read these properties from the record. The option `oci_config_in_record`, when set to true in the output config, will make the plugin read the mandatory and optional `oci_la` properties from the incoming record. The user must ensure that the necessary configs have been inserted using relevant filters, otherwise the respective chunk will be dropped. Below is an example to insert`oci_la_log_source_name` and `oci_la_log_group_id` in the record:
104
+
In case of multiple inputs, where `oci_la_*` properties can differ, you can add the properties in the record itself and instruct the plugin to read these properties from the record. The option `oci_config_in_record`, when set to `true` in the output config, will make the plugin read the mandatory and optional `oci_la` properties from the incoming record. The user must ensure that the necessary configurations have been inserted using relevant filters, otherwise the respective chunk will be dropped. The following example inserts`oci_la_log_source_name` and `oci_la_log_group_id` in the record:
114
105
115
106
{% tabs %}
116
107
{% tab title="fluent-bit.yaml" %}
@@ -127,7 +118,7 @@ pipeline:
127
118
add:
128
119
- oci_la_log_source_name <LOG_SOURCE_NAME>
129
120
- oci_la_log_group_id <LOG_GROUP_OCID>
130
-
121
+
131
122
outputs:
132
123
- name: oracle_log_analytics
133
124
match: '*'
@@ -186,10 +177,10 @@ pipeline:
186
177
profile_name: ADMIN
187
178
oci_la_log_source_name: example_log_source
188
179
oci_la_log_group_id: ocid.xxxxxx
189
-
oci_la_global_metadata:
180
+
oci_la_global_metadata:
190
181
- glob_key1 value1
191
182
- glob_key2 value2
192
-
oci_la_metadata:
183
+
oci_la_metadata:
193
184
- key1 value1
194
185
- key2 value2
195
186
tls: on
@@ -203,7 +194,7 @@ pipeline:
203
194
[INPUT]
204
195
Name dummy
205
196
Tag dummy
206
-
197
+
207
198
[Output]
208
199
Name oracle_log_analytics
209
200
Match *
@@ -223,7 +214,7 @@ pipeline:
223
214
{% endtab %}
224
215
{% endtabs %}
225
216
226
-
The above configuration will generate a payload that looks like this
217
+
The previous configuration will generate a payload that resembles the following:
227
218
228
219
```json
229
220
{
@@ -248,7 +239,7 @@ The above configuration will generate a payload that looks like this
248
239
249
240
The multiple `oci_la_global_metadata` and `oci_la_metadata` options are turned into a JSON object of key value pairs, nested under the key metadata.
250
241
251
-
With `oci_config_in_record` option set to true, the metadata key-value pairs will need to be injected in the record as an object of keyvalue pair nested under the respective metadata field. Below is an example of one such configuration
242
+
With `oci_config_in_record` option set to `true`, the metadata key/value pairs will need to be injected in the record as an object of key/value pairs nested under the respective metadata field. The following example shows one such configuration:
252
243
253
244
{% tabs %}
254
245
{% tab title="fluent-bit.yaml" %}
@@ -265,7 +256,7 @@ pipeline:
265
256
add:
266
257
- olgm.key1 val1
267
258
- olgm.key2 val2
268
-
259
+
269
260
- name: nest
270
261
match: '*'
271
262
operation: olgm.*
@@ -278,7 +269,7 @@ pipeline:
278
269
add:
279
270
- oci_la_log_source_name <LOG_SOURCE_NAME>
280
271
- oci_la_log_group_id <LOG_GROUP_OCID>
281
-
272
+
282
273
outputs:
283
274
- name: oracle_log_analytics
284
275
match: '*'
@@ -326,10 +317,10 @@ pipeline:
326
317
profile_name ADMIN
327
318
oci_config_in_record true
328
319
tls On
329
-
tls.verify Off
320
+
tls.verify Off
330
321
```
331
322
332
323
{% endtab %}
333
324
{% endtabs %}
334
325
335
-
The above configuration first injects the necessary metadata keys and values in the record directly, with a prefix olgm. attached to the keys in order to segregate the metadata keys from rest of the record keys. Then, using a nest filter only the metadata keys are selected by the filter and nested under `oci_la_global_metadata` key in the record, and the prefix `olgm`. is removed from the metadata keys.
326
+
The previous configuration first injects the necessary metadata keys and values in the record directly, with a prefix `olgm.` attached to the keys, which segregates the metadata keys from rest of the record keys. Then, using a `nest` filter only the metadata keys are selected by the filter and nested under `oci_la_global_metadata` key in the record, and the prefix `olgm.` is removed from the metadata keys.
0 commit comments