[Platform Observability] Create initial PO package for ingesting kibana ECS formatted logs #3622
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # newer versions go on top | ||
| - version: "0.0.1" | ||
| changes: | ||
| - description: Initial draft of the package | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"event":{"action":"http_request","category":["web"],"outcome":"unknown"},"http":{"request":{"method":"get"}},"url":{"domain":"localhost","path":"/internal/security/session","port":5601,"scheme":"http"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k="},"trace":{"id":"1c8c5808-d2d6-41fc-8cb7-998aa8996be9"},"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T12:05:03.742+00:00","message":"User is requesting [/internal/security/session] endpoint","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":7},"transaction":{"id":"f8863d86567119e6"}} | ||
| {"event":{"action":"http_request","category":["web"],"outcome":"unknown"},"http":{"request":{"method":"post"}},"url":{"domain":"localhost","path":"/internal/bsearch","port":5601,"query":"compress=true","scheme":"http"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k="},"trace":{"id":"abc8b4ad-5d96-42cf-9653-08aaeac0034e"},"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T12:05:08.178+00:00","message":"User is requesting [/internal/bsearch] endpoint","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":7},"transaction":{"id":"3098796995e24283"}} | ||
| {"event":{"action":"http_request","category":["web"],"outcome":"unknown"},"http":{"request":{"method":"post"}},"url":{"domain":"localhost","path":"/api/log_entries/summary","port":5601,"scheme":"http"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k="},"trace":{"id":"f4181218-b2d3-480e-b9da-78aef88683ff"},"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T12:05:08.187+00:00","message":"User is requesting [/api/log_entries/summary] endpoint","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":7},"transaction":{"id":"c1480039d6e6e321"}} | ||
| {"event":{"action":"http_request","category":["web"],"outcome":"unknown"},"http":{"request":{"method":"get"}},"url":{"domain":"localhost","path":"/internal/security/session","port":5601,"scheme":"http"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k="},"trace":{"id":"83d80454-6b8a-4727-91ba-22e6ab27e476"},"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T13:14:30.142+00:00","message":"User is requesting [/internal/security/session] endpoint","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":7},"transaction":{"id":"c1f684ff5fcf7eaf"}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| { | ||
| "expected": [ | ||
| { | ||
| "data_stream": { | ||
| "dataset": "kibana-audit-log", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-audit-log" | ||
| }, | ||
| "message": "{\"event\":{\"action\":\"http_request\",\"category\":[\"web\"],\"outcome\":\"unknown\"},\"http\":{\"request\":{\"method\":\"get\"}},\"url\":{\"domain\":\"localhost\",\"path\":\"/internal/security/session\",\"port\":5601,\"scheme\":\"http\"},\"user\":{\"name\":\"elastic\",\"roles\":[\"superuser\"]},\"kibana\":{\"space_id\":\"default\",\"session_id\":\"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k=\"},\"trace\":{\"id\":\"1c8c5808-d2d6-41fc-8cb7-998aa8996be9\"},\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2022-06-29T12:05:03.742+00:00\",\"message\":\"User is requesting [/internal/security/session] endpoint\",\"log\":{\"level\":\"INFO\",\"logger\":\"plugins.security.audit.ecs\"},\"process\":{\"pid\":7},\"transaction\":{\"id\":\"f8863d86567119e6\"}}" | ||
| }, | ||
| { | ||
| "data_stream": { | ||
| "dataset": "kibana-audit-log", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-audit-log" | ||
| }, | ||
| "message": "{\"event\":{\"action\":\"http_request\",\"category\":[\"web\"],\"outcome\":\"unknown\"},\"http\":{\"request\":{\"method\":\"post\"}},\"url\":{\"domain\":\"localhost\",\"path\":\"/internal/bsearch\",\"port\":5601,\"query\":\"compress=true\",\"scheme\":\"http\"},\"user\":{\"name\":\"elastic\",\"roles\":[\"superuser\"]},\"kibana\":{\"space_id\":\"default\",\"session_id\":\"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k=\"},\"trace\":{\"id\":\"abc8b4ad-5d96-42cf-9653-08aaeac0034e\"},\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2022-06-29T12:05:08.178+00:00\",\"message\":\"User is requesting [/internal/bsearch] endpoint\",\"log\":{\"level\":\"INFO\",\"logger\":\"plugins.security.audit.ecs\"},\"process\":{\"pid\":7},\"transaction\":{\"id\":\"3098796995e24283\"}}" | ||
| }, | ||
| { | ||
| "data_stream": { | ||
| "dataset": "kibana-audit-log", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-audit-log" | ||
| }, | ||
| "message": "{\"event\":{\"action\":\"http_request\",\"category\":[\"web\"],\"outcome\":\"unknown\"},\"http\":{\"request\":{\"method\":\"post\"}},\"url\":{\"domain\":\"localhost\",\"path\":\"/api/log_entries/summary\",\"port\":5601,\"scheme\":\"http\"},\"user\":{\"name\":\"elastic\",\"roles\":[\"superuser\"]},\"kibana\":{\"space_id\":\"default\",\"session_id\":\"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k=\"},\"trace\":{\"id\":\"f4181218-b2d3-480e-b9da-78aef88683ff\"},\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2022-06-29T12:05:08.187+00:00\",\"message\":\"User is requesting [/api/log_entries/summary] endpoint\",\"log\":{\"level\":\"INFO\",\"logger\":\"plugins.security.audit.ecs\"},\"process\":{\"pid\":7},\"transaction\":{\"id\":\"c1480039d6e6e321\"}}" | ||
matschaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| }, | ||
| { | ||
| "data_stream": { | ||
| "dataset": "kibana-audit-log", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-audit-log" | ||
| }, | ||
| "message": "{\"event\":{\"action\":\"http_request\",\"category\":[\"web\"],\"outcome\":\"unknown\"},\"http\":{\"request\":{\"method\":\"get\"}},\"url\":{\"domain\":\"localhost\",\"path\":\"/internal/security/session\",\"port\":5601,\"scheme\":\"http\"},\"user\":{\"name\":\"elastic\",\"roles\":[\"superuser\"]},\"kibana\":{\"space_id\":\"default\",\"session_id\":\"ccZ0sbxrmmJwo+/y2Mn1tmGIrKOuZYaF8voUh0SkA/k=\"},\"trace\":{\"id\":\"83d80454-6b8a-4727-91ba-22e6ab27e476\"},\"ecs\":{\"version\":\"8.0.0\"},\"@timestamp\":\"2022-06-29T13:14:30.142+00:00\",\"message\":\"User is requesting [/internal/security/session] endpoint\",\"log\":{\"level\":\"INFO\",\"logger\":\"plugins.security.audit.ecs\"},\"process\":{\"pid\":7},\"transaction\":{\"id\":\"c1f684ff5fcf7eaf\"}}" | ||
| } | ||
| ] | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| paths: | ||
| {{#each paths}} | ||
| - {{this}} | ||
| {{/each}} | ||
| processors: | ||
| - rename: | ||
| fields: | ||
| - from: "message" | ||
| to: "event.original" | ||
| - decode_json_fields: | ||
| fields: [event.original] | ||
| target: "" | ||
| add_error_key: true | ||
| overwrite_keys: true | ||
| {{#if processors}} | ||
| {{processors}} | ||
|
There was a problem hiding this comment. I'm guessing we can add "filebeat" processors (like https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html) to the manifest and have them show up here. Or maybe just put them in this file directly. Not sure what's in-fashion for packages today regarding reader-vs-ES side processing but personally I like to push as many simple transformations as I can to the collection point. Maybe @mtojek can advise. There was a problem hiding this comment. I did a quick test and it seems to work. I'll explore it a bit more. It would simplify the pipeline config There was a problem hiding this comment. It works.
this is all I have for the pipeline: There was a problem hiding this comment. A downside of this approach is that the tests are not able to use filebeat processor to parse the logs There was a problem hiding this comment. Another surprise here :) Hoping we can have some integration experts weigh-in. Are we maybe not supposed to use processors anymore? There was a problem hiding this comment. I used to follow the rule of thumb. Are you collecting large observability data and is the network loaded? If so, limit those with processors. There was a problem hiding this comment. We had a slack chat with @crespocarlos, and decided to go with the ingest pipeline due to code simplicity and convenient use of pipeline tests. There was a problem hiding this comment.
Interesting. Does the end user get to set processors? I'd expect network load to be a factor of deployment size. |
||
| {{/if}} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| --- | ||
matschaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| description: Pipeline for parsing ECS logs | ||
| processors: | ||
| - remove: | ||
| field: data_stream.dataset | ||
| ignore_missing: true | ||
| - remove: | ||
| field: event.dataset | ||
| ignore_missing: true | ||
| - set: | ||
| description: Uses event.dataset as a default for data_stream.dataset if the latter is not set. | ||
| field: data_stream.dataset | ||
| copy_from: event.dataset | ||
| if: ctx.event?.dataset instanceof String && ctx.event.dataset.length() > 1 | ||
| override: false | ||
| - script: | ||
| source: | | ||
| ctx.data_stream.dataset = /[\/*?"<>|, #:-]/.matcher(ctx.data_stream.dataset).replaceAll('_') | ||
| if: ctx.data_stream?.dataset != null | ||
| - script: | ||
| source: | | ||
| ctx.data_stream.namespace = /[\/*?"<>|, #:]/.matcher(ctx.data_stream.namespace).replaceAll('_') | ||
| if: ctx.data_stream?.namespace != null | ||
| - set: | ||
| field: data_stream.type | ||
| value: logs | ||
| - set: | ||
| field: data_stream.dataset | ||
| value: kibana-audit-log | ||
| override: false | ||
| - set: | ||
| field: data_stream.namespace | ||
| value: platform-observability | ||
| override: false | ||
| - set: | ||
| field: event.dataset | ||
| copy_from: data_stream.dataset | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| - name: data_stream.type | ||
| type: constant_keyword | ||
| description: Data stream type. | ||
| - name: data_stream.dataset | ||
| type: constant_keyword | ||
| description: Data stream dataset. | ||
| - name: data_stream.namespace | ||
| type: constant_keyword | ||
| description: Data stream namespace. | ||
| - name: "@timestamp" | ||
| type: date | ||
| description: Event timestamp. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # only used for tests | ||
| - name: message | ||
| external: ecs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| type: logs | ||
| title: Platform Observability Kibana Audit Logs | ||
| release: experimental | ||
| streams: | ||
| - input: logfile | ||
| vars: | ||
| - name: paths | ||
| type: text | ||
| title: Paths | ||
| multi: true | ||
| required: true | ||
| show_user: true | ||
| default: | ||
| - /tmp/service_logs/audit*.log | ||
| template_path: log.yml.hbs | ||
| title: Kibana Audit Logs | ||
| description: Collect Kibana Audit Logs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T11:24:17.898+00:00","message":"Trying to authenticate user request to /login.","log":{"level":"DEBUG","logger":"plugins.security.http"},"process":{"pid":7},"trace":{"id":"e6e1c25936546ec690b11a3b78b2a8db"},"transaction":{"id":"3be6994d7f6d5465"}} | ||
| {"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T11:24:17.898+00:00","message":"Authorization header is not presented.","log":{"level":"DEBUG","logger":"plugins.security.http"},"process":{"pid":7},"trace":{"id":"e6e1c25936546ec690b11a3b78b2a8db"},"transaction":{"id":"3be6994d7f6d5465"}} | ||
| {"ecs":{"version":"8.0.0"},"@timestamp":"2022-06-29T11:24:17.898+00:00","message":"Could not handle authentication attempt","log":{"level":"DEBUG","logger":"plugins.security.authentication"},"process":{"pid":7},"trace":{"id":"e6e1c25936546ec690b11a3b78b2a8db"},"transaction":{"id":"3be6994d7f6d5465"}} | ||
| {"_tag":"Right","right":"update_aliases_succeeded","ecs":{"version":"8.0.0"},"@timestamp":"2022-07-04T09:17:38.611+00:00","message":"[.kibana] MARK_VERSION_INDEX_READY RESPONSE","log":{"level":"DEBUG","logger":"savedobjects-service"},"process":{"pid":7},"trace":{"id":"a167d1124764379d4121b357e20baee2"},"transaction":{"id":"14717ae6e3b30d5a"}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| { | ||
| "expected": [ | ||
| { | ||
| "@timestamp": "2022-06-29T11:24:17.898+00:00", | ||
| "data_stream": { | ||
| "dataset": "kibana-logs", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.0.0" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-logs" | ||
| }, | ||
| "log": { | ||
| "level": "DEBUG", | ||
| "logger": "plugins.security.http" | ||
| }, | ||
| "message": "Trying to authenticate user request to /login.", | ||
| "process": { | ||
| "pid": 7 | ||
| }, | ||
| "trace": { | ||
| "id": "e6e1c25936546ec690b11a3b78b2a8db" | ||
| }, | ||
| "transaction": { | ||
| "id": "3be6994d7f6d5465" | ||
| } | ||
| }, | ||
| { | ||
| "@timestamp": "2022-06-29T11:24:17.898+00:00", | ||
| "data_stream": { | ||
| "dataset": "kibana-logs", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.0.0" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-logs" | ||
| }, | ||
| "log": { | ||
| "level": "DEBUG", | ||
| "logger": "plugins.security.http" | ||
| }, | ||
| "message": "Authorization header is not presented.", | ||
| "process": { | ||
| "pid": 7 | ||
| }, | ||
| "trace": { | ||
| "id": "e6e1c25936546ec690b11a3b78b2a8db" | ||
| }, | ||
| "transaction": { | ||
| "id": "3be6994d7f6d5465" | ||
| } | ||
| }, | ||
| { | ||
| "@timestamp": "2022-06-29T11:24:17.898+00:00", | ||
| "data_stream": { | ||
| "dataset": "kibana-logs", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.0.0" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-logs" | ||
| }, | ||
| "log": { | ||
| "level": "DEBUG", | ||
| "logger": "plugins.security.authentication" | ||
| }, | ||
| "message": "Could not handle authentication attempt", | ||
| "process": { | ||
| "pid": 7 | ||
| }, | ||
| "trace": { | ||
| "id": "e6e1c25936546ec690b11a3b78b2a8db" | ||
| }, | ||
| "transaction": { | ||
| "id": "3be6994d7f6d5465" | ||
| } | ||
| }, | ||
| { | ||
| "@timestamp": "2022-07-04T09:17:38.611+00:00", | ||
| "data_stream": { | ||
| "dataset": "kibana-logs", | ||
| "namespace": "platform-observability", | ||
| "type": "logs" | ||
| }, | ||
| "ecs": { | ||
| "version": "8.0.0" | ||
| }, | ||
| "event": { | ||
| "dataset": "kibana-logs" | ||
| }, | ||
| "log": { | ||
| "level": "DEBUG", | ||
| "logger": "savedobjects-service" | ||
| }, | ||
| "message": "[.kibana] MARK_VERSION_INDEX_READY RESPONSE", | ||
| "process": { | ||
| "pid": 7 | ||
| }, | ||
| "trace": { | ||
| "id": "a167d1124764379d4121b357e20baee2" | ||
| }, | ||
| "transaction": { | ||
| "id": "14717ae6e3b30d5a" | ||
| } | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| paths: | ||
| {{#each paths}} | ||
| - {{this}} | ||
| {{/each}} | ||
| {{#if processors}} | ||
| processors: | ||
| {{processors}} | ||
| {{/if}} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| --- | ||
| description: Pipeline for parsing ECS logs | ||
| processors: | ||
| - remove: | ||
| field: data_stream.dataset | ||
| ignore_missing: true | ||
| - remove: | ||
| field: event.dataset | ||
| ignore_missing: true | ||
| - set: | ||
| description: Uses event.dataset as a default for data_stream.dataset if the latter is not set. | ||
| field: data_stream.dataset | ||
| copy_from: event.dataset | ||
| if: ctx.event?.dataset instanceof String && ctx.event.dataset.length() > 1 | ||
| override: false | ||
| - script: | ||
| source: | | ||
| ctx.data_stream.dataset = /[\/*?"<>|, #:-]/.matcher(ctx.data_stream.dataset).replaceAll('_') | ||
| if: ctx.data_stream?.dataset != null | ||
| - script: | ||
| source: | | ||
| ctx.data_stream.namespace = /[\/*?"<>|, #:]/.matcher(ctx.data_stream.namespace).replaceAll('_') | ||
| if: ctx.data_stream?.namespace != null | ||
| - set: | ||
| field: data_stream.type | ||
| value: logs | ||
| - set: | ||
| field: data_stream.dataset | ||
| value: kibana-logs | ||
| override: false | ||
| - set: | ||
| field: data_stream.namespace | ||
| value: platform-observability | ||
| override: false | ||
| - set: | ||
| field: event.dataset | ||
| copy_from: data_stream.dataset | ||
| - rename: | ||
| field: message | ||
| target_field: _ecs_json_message | ||
| if: |- | ||
| def message = ctx.message; | ||
| return message != null | ||
| && message.startsWith('{') | ||
| && message.endsWith('}') | ||
| && message.contains('"@timestamp"') | ||
| ignore_missing: true | ||
| - json: | ||
| field: _ecs_json_message | ||
| add_to_root: true | ||
| add_to_root_conflict_strategy: merge | ||
| allow_duplicate_keys: true | ||
| if: ctx.containsKey('_ecs_json_message') | ||
| on_failure: | ||
| - rename: | ||
| field: _ecs_json_message | ||
| target_field: message | ||
| ignore_missing: true | ||
| - set: | ||
| field: error.message | ||
| value: Error while parsing JSON | ||
| override: false | ||
| - remove: | ||
| field: _ecs_json_message | ||
| ignore_missing: true | ||
| - remove: | ||
| field: _tag | ||
| ignore_missing: true | ||
| - remove: | ||
| field: right | ||
| ignore_missing: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| - name: data_stream.type | ||
| type: constant_keyword | ||
| description: Data stream type. | ||
| - name: data_stream.dataset | ||
| type: constant_keyword | ||
| description: Data stream dataset. | ||
| - name: data_stream.namespace | ||
| type: constant_keyword | ||
| description: Data stream namespace. | ||
| - name: "@timestamp" | ||
| type: date | ||
| description: Event timestamp. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # only used for tests | ||
| - name: ecs.version | ||
| external: ecs | ||
| - name: log.level | ||
| external: ecs | ||
| - name: log.logger | ||
| external: ecs | ||
| - name: message | ||
| external: ecs | ||
| - name: process.pid | ||
| external: ecs | ||
| - name: trace.id | ||
| external: ecs | ||
| - name: transaction.id | ||
| external: ecs |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.