From 6a1979e28d55debbbdd75ac3cb59c18d9d5aa371 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 5 Feb 2025 13:32:48 +0100 Subject: [PATCH 01/27] audit logging initial changes --- ...elating-kibana-elasticsearch-audit-logs.md | 354 +-------------- .../elasticsearch-audit-events.md | 4 +- ...-audit-logs-in-orchestrated-deployments.md | 24 - .../enabling-audit-logs.md | 171 ++++++++ .../enabling-elasticsearch-audit-logs.md | 35 -- .../enabling-kibana-audit-logs.md | 409 ------------------ .../security-event-audit-logging.md | 24 +- deploy-manage/toc.yml | 4 +- .../cloud-on-k8s/k8s_audit_logging.md | 40 -- .../cloud-enterprise/ece-enable-auditing.md | 33 -- .../cloud/ec-enable-logging-and-monitoring.md | 1 + raw-migrated-files/toc.yml | 2 - 12 files changed, 199 insertions(+), 902 deletions(-) delete mode 100644 deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md create mode 100644 deploy-manage/monitor/logging-configuration/enabling-audit-logs.md delete mode 100644 deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md delete mode 100644 deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md delete mode 100644 raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md delete mode 100644 raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index 467d76aa52..dff13687a1 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -8,359 +8,7 @@ applies: stack: all --- -# Correlating Kibana and Elasticsearch audit logs [xpack-security-audit-logging] - -Audit logging is a [subscription feature](https://www.elastic.co/subscriptions) that you can enable to keep track of security-related events, such as authorization success and failures. Logging these events enables you to monitor {{kib}} for suspicious activity and provides evidence in the event of an attack. - -Use the {{kib}} audit logs in conjunction with [{{es}} audit logging](enabling-elasticsearch-audit-logs.md) to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. For more information on enabling audit logging in {{es}}, refer to [Auditing security events](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing.html). - -::::{note} -Audit logs are **disabled** by default. To enable this functionality, you must set `xpack.security.audit.enabled` to `true` in `kibana.yml`. - -You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [Audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). - -:::: - - -## Audit events [xpack-security-ecs-audit-logging] - -Refer to the table of events that can be logged for auditing purposes. - -Each event is broken down into [category](enabling-kibana-audit-logs.md#field-event-category), [type](enabling-kibana-audit-logs.md#field-event-type), [action](enabling-kibana-audit-logs.md#field-event-action) and [outcome](enabling-kibana-audit-logs.md#field-event-outcome) fields to make it easy to filter, query and aggregate the resulting logs. The [trace.id](enabling-kibana-audit-logs.md#field-trace-id) field can be used to correlate multiple events that originate from the same request. - -Refer to [Audit schema](enabling-kibana-audit-logs.md#xpack-security-ecs-audit-schema) for a table of fields that get logged with audit event. - -::::{note} -To ensure that a record of every operation is persisted even in case of an unexpected error, asynchronous write operations are logged immediately after all authorization checks have passed, but before the response from {{es}} is received. Refer to the corresponding {{es}} logs for potential write errors. - -:::: - - -| | -| --- | -| #### Category: authentication [_category_authentication]


| -| **Action** | **Outcome** | **Description** | -| `user_login` | `success` | User has logged in successfully. | -| `failure` | Failed login attempt (e.g. due to invalid credentials). | -| `user_logout` | `unknown` | User is logging out. | -| `session_cleanup` | `unknown` | Removing invalid or expired session. | -| `access_agreement_acknowledged` | n/a | User has acknowledged the access agreement. | -| #### Category: database [_category_database]

##### Type: creation [_type_creation]



| -| **Action** | **Outcome** | **Description** | -| `saved_object_create` | `unknown` | User is creating a saved object. | -| `failure` | User is not authorized to create a saved object. | -| `saved_object_open_point_in_time` | `unknown` | User is creating a Point In Time to use when querying saved objects. | -| `failure` | User is not authorized to create a Point In Time for the provided saved object types. | -| `connector_create` | `unknown` | User is creating a connector. | -| `failure` | User is not authorized to create a connector. | -| `rule_create` | `unknown` | User is creating a rule. | -| `failure` | User is not authorized to create a rule. | -| `ad_hoc_run_create` | `unknown` | User is creating an ad hoc run. | -| `failure` | User is not authorized to create an ad hoc run. | -| `space_create` | `unknown` | User is creating a space. | -| `failure` | User is not authorized to create a space. | -| `case_create` | `unknown` | User is creating a case. | -| `failure` | User is not authorized to create a case. | -| `case_configuration_create` | `unknown` | User is creating a case configuration. | -| `failure` | User is not authorized to create a case configuration. | -| `case_comment_create` | `unknown` | User is creating a case comment. | -| `failure` | User is not authorized to create a case comment. | -| `case_comment_bulk_create` | `unknown` | User is creating multiple case comments. | -| `failure` | User is not authorized to create multiple case comments. | -| `case_user_action_create_comment` | `success` | User has created a case comment. | -| `case_user_action_create_case` | `success` | User has created a case. | -| `ml_put_ad_job` | `success` | Creating anomaly detection job. | -| `failure` | Failed to create anomaly detection job. | -| `ml_put_ad_datafeed` | `success` | Creating anomaly detection datafeed. | -| `failure` | Failed to create anomaly detection datafeed. | -| `ml_put_calendar` | `success` | Creating calendar. | -| `failure` | Failed to create calendar. | -| `ml_post_calendar_events` | `success` | Adding events to calendar. | -| `failure` | Failed to add events to calendar. | -| `ml_forecast` | `success` | Creating anomaly detection forecast. | -| `failure` | Failed to create anomaly detection forecast. | -| `ml_put_filter` | `success` | Creating filter. | -| `failure` | Failed to create filter. | -| `ml_put_dfa_job` | `success` | Creating data frame analytics job. | -| `failure` | Failed to create data frame analytics job. | -| `ml_put_trained_model` | `success` | Creating trained model. | -| `failure` | Failed to create trained model. | -| `product_documentation_create` | `unknown` | User requested to install the product documentation for use in AI Assistants. | -| `knowledge_base_entry_create` | `success` | User has created knowledge base entry [id=x] | -| `failure` | Failed attempt to create a knowledge base entry | -| `knowledge_base_entry_update` | `success` | User has updated knowledge base entry [id=x] | -| `failure` | Failed attempt to update a knowledge base entry | -| `knowledge_base_entry_delete` | `success` | User has deleted knowledge base entry [id=x] | -| `failure` | Failed attempt to delete a knowledge base entry | -| ##### Type: change [_type_change]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_update` | `unknown` | User is updating a saved object. | -| `failure` | User is not authorized to update a saved object. | -| `saved_object_update_objects_spaces` | `unknown` | User is adding and/or removing a saved object to/from other spaces. | -| `failure` | User is not authorized to add or remove a saved object to or from other spaces. | -| `saved_object_remove_references` | `unknown` | User is removing references to a saved object. | -| `failure` | User is not authorized to remove references to a saved object. | -| `saved_object_collect_multinamespace_references` | `success` | User has accessed references to a multi-space saved object. | -| `failure` | User is not authorized to access references to a multi-space saved object. | -| `connector_update` | `unknown` | User is updating a connector. | -| `failure` | User is not authorized to update a connector. | -| `rule_update` | `unknown` | User is updating a rule. | -| `failure` | User is not authorized to update a rule. | -| `rule_update_api_key` | `unknown` | User is updating the API key of a rule. | -| `failure` | User is not authorized to update the API key of a rule. | -| `rule_enable` | `unknown` | User is enabling a rule. | -| `failure` | User is not authorized to enable a rule. | -| `rule_disable` | `unknown` | User is disabling a rule. | -| `failure` | User is not authorized to disable a rule. | -| `rule_mute` | `unknown` | User is muting a rule. | -| `failure` | User is not authorized to mute a rule. | -| `rule_unmute` | `unknown` | User is unmuting a rule. | -| `failure` | User is not authorized to unmute a rule. | -| `rule_alert_mute` | `unknown` | User is muting an alert. | -| `failure` | User is not authorized to mute an alert. | -| `rule_alert_unmute` | `unknown` | User is unmuting an alert. | -| `failure` | User is not authorized to unmute an alert. | -| `space_update` | `unknown` | User is updating a space. | -| `failure` | User is not authorized to update a space. | -| `alert_update` | `unknown` | User is updating an alert. | -| `failure` | User is not authorized to update an alert. | -| `rule_snooze` | `unknown` | User is snoozing a rule. | -| `failure` | User is not authorized to snooze a rule. | -| `rule_unsnooze` | `unknown` | User is unsnoozing a rule. | -| `failure` | User is not authorized to unsnooze a rule. | -| `case_update` | `unknown` | User is updating a case. | -| `failure` | User is not authorized to update a case. | -| `case_push` | `unknown` | User is pushing a case to an external service. | -| `failure` | User is not authorized to push a case to an external service. | -| `case_configuration_update` | `unknown` | User is updating a case configuration. | -| `failure` | User is not authorized to update a case configuration. | -| `case_comment_update` | `unknown` | User is updating a case comment. | -| `failure` | User is not authorized to update a case comment. | -| `case_user_action_add_case_assignees` | `success` | User has added a case assignee. | -| `case_user_action_update_case_connector` | `success` | User has updated a case connector. | -| `case_user_action_update_case_description` | `success` | User has updated a case description. | -| `case_user_action_update_case_settings` | `success` | User has updated the case settings. | -| `case_user_action_update_case_severity` | `success` | User has updated the case severity. | -| `case_user_action_update_case_status` | `success` | User has updated the case status. | -| `case_user_action_pushed_case` | `success` | User has pushed a case to an external service. | -| `case_user_action_add_case_tags` | `success` | User has added tags to a case. | -| `case_user_action_update_case_title` | `success` | User has updated the case title. | -| `ml_open_ad_job` | `success` | Opening anomaly detection job. | -| `failure` | Failed to open anomaly detection job. | -| `ml_close_ad_job` | `success` | Closing anomaly detection job. | -| `failure` | Failed to close anomaly detection job. | -| `ml_start_ad_datafeed` | `success` | Starting anomaly detection datafeed. | -| `failure` | Failed to start anomaly detection datafeed. | -| `ml_stop_ad_datafeed` | `success` | Stopping anomaly detection datafeed. | -| `failure` | Failed to stop anomaly detection datafeed. | -| `ml_update_ad_job` | `success` | Updating anomaly detection job. | -| `failure` | Failed to update anomaly detection job. | -| `ml_reset_ad_job` | `success` | Resetting anomaly detection job. | -| `failure` | Failed to reset anomaly detection job. | -| `ml_revert_ad_snapshot` | `success` | Reverting anomaly detection snapshot. | -| `failure` | Failed to revert anomaly detection snapshot. | -| `ml_update_ad_datafeed` | `success` | Updating anomaly detection datafeed. | -| `failure` | Failed to update anomaly detection datafeed. | -| `ml_put_calendar_job` | `success` | Adding job to calendar. | -| `failure` | Failed to add job to calendar. | -| `ml_delete_calendar_job` | `success` | Removing job from calendar. | -| `failure` | Failed to remove job from calendar. | -| `ml_update_filter` | `success` | Updating filter. | -| `failure` | Failed to update filter. | -| `ml_start_dfa_job` | `success` | Starting data frame analytics job. | -| `failure` | Failed to start data frame analytics job. | -| `ml_stop_dfa_job` | `success` | Stopping data frame analytics job. | -| `failure` | Failed to stop data frame analytics job. | -| `ml_update_dfa_job` | `success` | Updating data frame analytics job. | -| `failure` | Failed to update data frame analytics job. | -| `ml_start_trained_model_deployment` | `success` | Starting trained model deployment. | -| `failure` | Failed to start trained model deployment. | -| `ml_stop_trained_model_deployment` | `success` | Stopping trained model deployment. | -| `failure` | Failed to stop trained model deployment. | -| `ml_update_trained_model_deployment` | `success` | Updating trained model deployment. | -| `failure` | Failed to update trained model deployment. | -| `product_documentation_update` | `unknown` | User requested to update the product documentation for use in AI Assistants. | -| ##### Type: deletion [_type_deletion]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_delete` | `unknown` | User is deleting a saved object. | -| `failure` | User is not authorized to delete a saved object. | -| `saved_object_close_point_in_time` | `unknown` | User is deleting a Point In Time that was used to query saved objects. | -| `failure` | User is not authorized to delete a Point In Time. | -| `connector_delete` | `unknown` | User is deleting a connector. | -| `failure` | User is not authorized to delete a connector. | -| `rule_delete` | `unknown` | User is deleting a rule. | -| `failure` | User is not authorized to delete a rule. | -| `ad_hoc_run_delete` | `unknown` | User is deleting an ad hoc run. | -| `failure` | User is not authorized to delete an ad hoc run. | -| `space_delete` | `unknown` | User is deleting a space. | -| `failure` | User is not authorized to delete a space. | -| `case_delete` | `unknown` | User is deleting a case. | -| `failure` | User is not authorized to delete a case. | -| `case_comment_delete_all` | `unknown` | User is deleting all comments associated with a case. | -| `failure` | User is not authorized to delete all comments associated with a case. | -| `case_comment_delete` | `unknown` | User is deleting a case comment. | -| `failure` | User is not authorized to delete a case comment. | -| `case_user_action_delete_case_assignees` | `success` | User has removed a case assignee. | -| `case_user_action_delete_comment` | `success` | User has deleted a case comment. | -| `case_user_action_delete_case` | `success` | User has deleted a case. | -| `case_user_action_delete_case_tags` | `success` | User has removed tags from a case. | -| `ml_delete_ad_job` | `success` | Deleting anomaly detection job. | -| `failure` | Failed to delete anomaly detection job. | -| `ml_delete_model_snapshot` | `success` | Deleting model snapshot. | -| `failure` | Failed to delete model snapshot. | -| `ml_delete_ad_datafeed` | `success` | Deleting anomaly detection datafeed. | -| `failure` | Failed to delete anomaly detection datafeed. | -| `ml_delete_calendar` | `success` | Deleting calendar. | -| `failure` | Failed to delete calendar. | -| `ml_delete_calendar_event` | `success` | Deleting calendar event. | -| `failure` | Failed to delete calendar event. | -| `ml_delete_filter` | `success` | Deleting filter. | -| `failure` | Failed to delete filter. | -| `ml_delete_forecast` | `success` | Deleting forecast. | -| `failure` | Failed to delete forecast. | -| `ml_delete_dfa_job` | `success` | Deleting data frame analytics job. | -| `failure` | Failed to delete data frame analytics job. | -| `ml_delete_trained_model` | `success` | Deleting trained model. | -| `failure` | Failed to delete trained model. | -| `product_documentation_delete` | `unknown` | User requested to delete the product documentation for use in AI Assistants. | -| ##### Type: access [_type_access]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_get` | `success` | User has accessed a saved object. | -| `failure` | User is not authorized to access a saved object. | -| `saved_object_resolve` | `success` | User has accessed a saved object. | -| `failure` | User is not authorized to access a saved object. | -| `saved_object_find` | `success` | User has accessed a saved object as part of a search operation. | -| `failure` | User is not authorized to search for saved objects. | -| `connector_get` | `success` | User has accessed a connector. | -| `failure` | User is not authorized to access a connector. | -| `connector_find` | `success` | User has accessed a connector as part of a search operation. | -| `failure` | User is not authorized to search for connectors. | -| `rule_get` | `success` | User has accessed a rule. | -| `failure` | User is not authorized to access a rule. | -| `rule_get_execution_log` | `success` | User has accessed execution log for a rule. | -| `failure` | User is not authorized to access execution log for a rule. | -| `rule_find` | `success` | User has accessed a rule as part of a search operation. | -| `failure` | User is not authorized to search for rules. | -| `rule_schedule_backfill` | `success` | User has accessed a rule as part of a backfill schedule operation. | -| `failure` | User is not authorized to access rule for backfill scheduling. | -| `ad_hoc_run_get` | `success` | User has accessed an ad hoc run. | -| `failure` | User is not authorized to access ad hoc run. | -| `ad_hoc_run_find` | `success` | User has accessed an ad hoc run as part of a search operation. | -| `failure` | User is not authorized to search for ad hoc runs. | -| `space_get` | `success` | User has accessed a space. | -| `failure` | User is not authorized to access a space. | -| `space_find` | `success` | User has accessed a space as part of a search operation. | -| `failure` | User is not authorized to search for spaces. | -| `alert_get` | `success` | User has accessed an alert. | -| `failure` | User is not authorized to access an alert. | -| `alert_find` | `success` | User has accessed an alert as part of a search operation. | -| `failure` | User is not authorized to access alerts. | -| `case_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_bulk_get` | `success` | User has accessed multiple cases. | -| `failure` | User is not authorized to access multiple cases. | -| `case_resolve` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_find` | `success` | User has accessed a case as part of a search operation. | -| `failure` | User is not authorized to search for cases. | -| `case_ids_by_alert_id_get` | `success` | User has accessed cases. | -| `failure` | User is not authorized to access cases. | -| `case_get_metrics` | `success` | User has accessed metrics for a case. | -| `failure` | User is not authorized to access metrics for a case. | -| `cases_get_metrics` | `success` | User has accessed metrics for cases. | -| `failure` | User is not authorized to access metrics for cases. | -| `case_configuration_find` | `success` | User has accessed a case configuration as part of a search operation. | -| `failure` | User is not authorized to search for case configurations. | -| `case_comment_get_metrics` | `success` | User has accessed metrics for case comments. | -| `failure` | User is not authorized to access metrics for case comments. | -| `case_comment_alerts_attach_to_case` | `success` | User has accessed case alerts. | -| `failure` | User is not authorized to access case alerts. | -| `case_comment_get` | `success` | User has accessed a case comment. | -| `failure` | User is not authorized to access a case comment. | -| `case_comment_bulk_get` | `success` | User has accessed multiple case comments. | -| `failure` | User is not authorized to access multiple case comments. | -| `case_comment_get_all` | `success` | User has accessed case comments. | -| `failure` | User is not authorized to access case comments. | -| `case_comment_find` | `success` | User has accessed a case comment as part of a search operation. | -| `failure` | User is not authorized to search for case comments. | -| `case_categories_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_tags_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_reporters_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_find_statuses` | `success` | User has accessed a case as part of a search operation. | -| `failure` | User is not authorized to search for cases. | -| `case_user_actions_get` | `success` | User has accessed the user activity of a case. | -| `failure` | User is not authorized to access the user activity of a case. | -| `case_user_actions_find` | `success` | User has accessed the user activity of a case as part of a search operation. | -| `failure` | User is not authorized to access the user activity of a case. | -| `case_user_action_get_metrics` | `success` | User has accessed metrics for the user activity of a case. | -| `failure` | User is not authorized to access metrics for the user activity of a case. | -| `case_user_action_get_users` | `success` | User has accessed the users associated with a case. | -| `failure` | User is not authorized to access the users associated with a case. | -| `case_connectors_get` | `success` | User has accessed the connectors of a case. | -| `failure` | User is not authorized to access the connectors of a case. | -| `ml_infer_trained_model` | `success` | Inferring using trained model. | -| `failure` | Failed to infer using trained model. | -| #### Category: web [_category_web]


| -| **Action** | **Outcome** | **Description** | -| `http_request` | `unknown` | User is making an HTTP request. | - - -## Audit schema [xpack-security-ecs-audit-schema] - -Audit logs are written in JSON using [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/1.6/index.html) specification. - -| | -| --- | -| #### Base Fields [_base_fields]


| -| **Field** | **Description** | -| `@timestamp` | Time when the event was generated.
Example: `2016-05-23T08:05:34.853Z` | -| `message` | Human readable description of the event. | -| #### Event Fields [_event_fields]


| -| **Field** | **Description** | -| $$$field-event-action$$$ `event.action` | The action captured by the event.
Refer to [Audit events](enabling-kibana-audit-logs.md#xpack-security-ecs-audit-logging) for a table of possible actions. | -| $$$field-event-category$$$ `event.category` | High level category associated with the event.
This field is closely related to `event.type`, which is used as a subcategory.
Possible values:`database`,`web`,`authentication` | -| $$$field-event-type$$$ `event.type` | Subcategory associated with the event.
This field can be used along with the `event.category` field to enable filtering events down to a level appropriate for single visualization.
Possible values:`creation`,`access`,`change`,`deletion` | -| $$$field-event-outcome$$$ `event.outcome` | Denotes whether the event represents a success or failure:

* Any actions that the user is not authorized to perform are logged with outcome: `failure`
* Authorized read operations are only logged after successfully fetching the data from {{es}} with outcome: `success`
* Authorized create, update, or delete operations are logged before attempting the operation in {{es}} with outcome: `unknown`

Possible values: `success`, `failure`, `unknown`
| -| #### User Fields [_user_fields]


| -| **Field** | **Description** | -| `user.id` | Unique identifier of the user across sessions (See [user profiles](../../users-roles/cluster-or-deployment-auth/user-profiles.md)). | -| `user.name` | Login name of the user.
Example: `jdoe` | -| `user.roles[]` | Set of user roles at the time of the event.
Example: `[kibana_admin, reporting_user]` | -| #### Kibana Fields [_kibana_fields]


| -| **Field** | **Description** | -| `kibana.space_id` | ID of the space associated with the event.
Example: `default` | -| `kibana.session_id` | ID of the user session associated with the event.
Each login attempt results in a unique session id. | -| `kibana.saved_object.type` | Type of saved object associated with the event.
Example: `dashboard` | -| `kibana.saved_object.id` | ID of the saved object associated with the event. | -| `kibana.authentication_provider` | Name of the authentication provider associated with the event.
Example: `my-saml-provider` | -| `kibana.authentication_type` | Type of the authentication provider associated with the event.
Example: `saml` | -| `kibana.authentication_realm` | Name of the Elasticsearch realm that has authenticated the user.
Example: `native` | -| `kibana.lookup_realm` | Name of the Elasticsearch realm where the user details were retrieved from.
Example: `native` | -| `kibana.add_to_spaces[]` | Set of space IDs that a saved object is being shared to as part of the event.
Example: `[default, marketing]` | -| `kibana.delete_from_spaces[]` | Set of space IDs that a saved object is being removed from as part of the event.
Example: `[marketing]` | -| #### Error Fields [_error_fields]


| -| **Field** | **Description** | -| `error.code` | Error code describing the error. | -| `error.message` | Error message. | -| #### HTTP and URL Fields [_http_and_url_fields]


| -| **Field** | **Description** | -| `client.ip` | Client IP address. | -| `http.request.method` | HTTP request method.
Example: `get`, `post`, `put`, `delete` | -| `http.request.headers.x-forwarded-for` | `X-Forwarded-For` request header used to identify the originating client IP address when connecting through proxy servers.
Example: `161.66.20.177, 236.198.214.101` | -| `url.domain` | Domain of the URL.
Example: `www.elastic.co` | -| `url.path` | Path of the request.
Example: `/search` | -| `url.port` | Port of the request.
Example: `443` | -| `url.query` | The query field describes the query string of the request.
Example: `q=elasticsearch` | -| `url.scheme` | Scheme of the request.
Example: `https` | -| #### Tracing Fields [_tracing_fields]


| -| **Field** | **Description** | -| $$$field-trace-id$$$ `trace.id` | Unique identifier allowing events of the same transaction from {{kib}} and {{es}} to be correlated. | - - -## Correlating audit events [xpack-security-ecs-audit-correlation] +# Correlating Kibana and Elasticsearch audit events [xpack-security-ecs-audit-correlation] Audit events can be correlated in two ways: diff --git a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md index cbca6ce141..7bcf440eed 100644 --- a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md +++ b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md @@ -8,9 +8,11 @@ applies: stack: all --- +% Part of this doc should probably go to reference content at a later stage + # Elasticsearch audit events [audit-event-types] -When you are [auditing security events](enabling-elasticsearch-audit-logs.md), a single client request might generate multiple audit events, across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. +When you are [auditing security events](enabling-audit-logs.md), a single client request might generate multiple audit events, across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. Use the [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) setting in `elasticsearch.yml` to specify the kind of events you want to include in the auditing output. diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md deleted file mode 100644 index 06e2ffd2d8..0000000000 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -mapped_urls: - - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-enable-auditing.html - - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_audit_logging.html - - https://www.elastic.co/guide/en/cloud/current/ec-enable-logging-and-monitoring.html#ec-enable-audit-logs -applies: - hosted: all - ece: all - eck: all ---- - -# Enabling audit logs in orchestrated deployments - -% What needs to be done: Refine - -% GitHub issue: https://github.com/elastic/docs-projects/issues/350 - -% Scope notes: Merge the content and even consider putting everything under a global section that also covers Elasticsearch self-managed - -% Use migrated content from existing pages that map to this page: - -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md new file mode 100644 index 0000000000..82a4305a62 --- /dev/null +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -0,0 +1,171 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/reference/current/enable-audit-logging.html + - https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html + - https://www.elastic.co/guide/en/cloud-enterprise/current/ece-enable-auditing.html + - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_audit_logging.html + - https://www.elastic.co/guide/en/cloud/current/ec-enable-logging-and-monitoring.html#ec-enable-audit-logs +applies: + hosted: all + ece: all + eck: all + stack: all +--- + +% What needs to be done: Refine + +% GitHub issue: https://github.com/elastic/docs-projects/issues/350 + +% Scope notes: Merge the content and even consider putting everything under a global section that also covers Elasticsearch self-managed (done) + +% Use migrated content from existing pages that map to this page: + +% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md +% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md +% - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md + +# Enable audit logging [enable-audit-logging] + +You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). + +Audit logging also provides forensic evidence in the event of an attack. + +::::{important} +Audit logs are **disabled** by default. You must explicitly enable audit logging. +:::: + +## Enable audit logging in Elasticsearch [enable-audit-logging-elasticsearch] + +To enable audit logging: + +1. Set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. +2. Restart {{es}}. + +When audit logging is enabled, [security events](elasticsearch-audit-events.md) are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, see [Logfile audit output](logfile-audit-output.md). + +::::{note} +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). +:::: + +## Enable audit logging in Kibana [enable-audit-logging-kibana] + +To enable audit logging in {{kib}}: + +1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml` +2. Restart {{kib}} + +::::{note} +You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). +:::: + +## Enable audit logging in orchestrated deployments [enable-audit-logging-orchestrated] + +In orchestrated systems you just need to set the relevant setting through the orchestrator, as with any other {{es}} or {{kib}} setting. + +Below you have some examples: + +::::::{tab-set} + +:::::{tab-item} Elastic Cloud Hosted + +To enable audit logs on your deployment: + +1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). +2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. + + ::::{tip} + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + :::: + +3. From your deployment menu, go to the **Edit** page. +4. To enable audit logs in {{es}}, in the **Elasticsearch** section select **Manage user settings and extensions**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for each node instead. +5. To enable audit logs in {{kib}}, in the **Kibana** section select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. +6. Add `xpack.security.audit.enabled: true` to the user settings. +7. Select **Save changes**. + +A plan change will run on your deployment. When it finishes, audit logs will be delivered to your monitoring deployment. + +::::: + +:::::{tab-item} ECE +In Elastic Cloud Enterprise, to get audit events for both Elasticsearch and Kibana, you need to enable auditing for each component separately. + +To enable auditing for Elasticsearch: + +1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the **Deployments** page, select your deployment. + ::::{tip} + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + :::: +3. From your deployment menu, go to the **Edit** page. +4. In the **Elasticsearch** section, select **Edit user settings and plugins**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for the first node instead. +5. Add the setting `xpack.security.audit.enabled: true`. +6. Select **Save**. + +For more information and other available auditing settings in Elasticsearch, check [Auditing security settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). Audit logs can be viewed within Elasticsearch logs. + +To enable auditing for Kibana: + +1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +2. On the **Deployments** page, select your deployment. + ::::{tip} + On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. + :::: +3. From your deployment menu, go to the **Edit** page. +4. In the **Kibana** section, select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. +5. Add the setting `xpack.security.audit.enabled: true`. +6. If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. +7. Select **Save**. + +For more information about audit logging in Kibana, check [Audit Logging](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html). +::::: + +:::::{tab-item} ECK + +Audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section when audit logging is enabled (it is disabled by default). + +```yaml +apiVersion: elasticsearch.k8s.elastic.co/v1 +kind: Elasticsearch +spec: + monitoring: + metrics: + elasticsearchRefs: + - name: monitoring + namespace: observability + logs: + elasticsearchRefs: + - name: monitoring + namespace: observability + nodeSets: + - name: default + config: + # https://www.elastic.co/guide/en/elasticsearch/reference/current/enable-audit-logging.html + xpack.security.audit.enabled: true +--- +apiVersion: kibana.k8s.elastic.co/v1 +kind: Kibana +spec: + monitoring: + metrics: + elasticsearchRefs: + - name: monitoring + namespace: observability + logs: + elasticsearchRefs: + - name: monitoring + namespace: observability + config: + # https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html + xpack.security.audit.enabled: true +``` + +::::: + +:::::: + + +Orchestrated deployments allow to change certain settings regarding audit logging, such as: + +* asdad +* bsdsda diff --git a/deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md deleted file mode 100644 index 462a6ef470..0000000000 --- a/deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/enable-audit-logging.html -applies: - hosted: all - ece: all - eck: all - stack: all ---- - -# Enabling elasticsearch audit logs [enable-audit-logging] - -You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). - -Audit logging also provides forensic evidence in the event of an attack. - -::::{important} -Audit logs are **disabled** by default. You must explicitly enable audit logging. - -:::: - - -::::{tip} -Audit logs are only available on certain subscription levels. For more information, see {{subscriptions}}. -:::: - - -To enable audit logging: - -1. Set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. -2. Restart {{es}}. - -When audit logging is enabled, [security events](elasticsearch-audit-events.md) are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, see [Logfile audit output](logfile-audit-output.md). - -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, see [Auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). diff --git a/deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md deleted file mode 100644 index e3ace4b82f..0000000000 --- a/deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md +++ /dev/null @@ -1,409 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html -applies: - hosted: all - ece: all - eck: all - stack: all ---- - -# Enabling Kibana audit logs [xpack-security-audit-logging] - -Audit logging is a [subscription feature](https://www.elastic.co/subscriptions) that you can enable to keep track of security-related events, such as authorization success and failures. Logging these events enables you to monitor {{kib}} for suspicious activity and provides evidence in the event of an attack. - -Use the {{kib}} audit logs in conjunction with [{{es}} audit logging](enabling-elasticsearch-audit-logs.md) to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. For more information on enabling audit logging in {{es}}, refer to [Auditing security events](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing.html). - -::::{note} -Audit logs are **disabled** by default. To enable this functionality, you must set `xpack.security.audit.enabled` to `true` in `kibana.yml`. - -You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [Audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). - -:::: - - -## Audit events [xpack-security-ecs-audit-logging] - -Refer to the table of events that can be logged for auditing purposes. - -Each event is broken down into [category](#field-event-category), [type](#field-event-type), [action](#field-event-action) and [outcome](#field-event-outcome) fields to make it easy to filter, query and aggregate the resulting logs. The [trace.id](#field-trace-id) field can be used to correlate multiple events that originate from the same request. - -Refer to [Audit schema](#xpack-security-ecs-audit-schema) for a table of fields that get logged with audit event. - -::::{note} -To ensure that a record of every operation is persisted even in case of an unexpected error, asynchronous write operations are logged immediately after all authorization checks have passed, but before the response from {{es}} is received. Refer to the corresponding {{es}} logs for potential write errors. - -:::: - - -| | -| --- | -| #### Category: authentication [_category_authentication]


| -| **Action** | **Outcome** | **Description** | -| `user_login` | `success` | User has logged in successfully. | -| `failure` | Failed login attempt (e.g. due to invalid credentials). | -| `user_logout` | `unknown` | User is logging out. | -| `session_cleanup` | `unknown` | Removing invalid or expired session. | -| `access_agreement_acknowledged` | n/a | User has acknowledged the access agreement. | -| #### Category: database [_category_database]

##### Type: creation [_type_creation]



| -| **Action** | **Outcome** | **Description** | -| `saved_object_create` | `unknown` | User is creating a saved object. | -| `failure` | User is not authorized to create a saved object. | -| `saved_object_open_point_in_time` | `unknown` | User is creating a Point In Time to use when querying saved objects. | -| `failure` | User is not authorized to create a Point In Time for the provided saved object types. | -| `connector_create` | `unknown` | User is creating a connector. | -| `failure` | User is not authorized to create a connector. | -| `rule_create` | `unknown` | User is creating a rule. | -| `failure` | User is not authorized to create a rule. | -| `ad_hoc_run_create` | `unknown` | User is creating an ad hoc run. | -| `failure` | User is not authorized to create an ad hoc run. | -| `space_create` | `unknown` | User is creating a space. | -| `failure` | User is not authorized to create a space. | -| `case_create` | `unknown` | User is creating a case. | -| `failure` | User is not authorized to create a case. | -| `case_configuration_create` | `unknown` | User is creating a case configuration. | -| `failure` | User is not authorized to create a case configuration. | -| `case_comment_create` | `unknown` | User is creating a case comment. | -| `failure` | User is not authorized to create a case comment. | -| `case_comment_bulk_create` | `unknown` | User is creating multiple case comments. | -| `failure` | User is not authorized to create multiple case comments. | -| `case_user_action_create_comment` | `success` | User has created a case comment. | -| `case_user_action_create_case` | `success` | User has created a case. | -| `ml_put_ad_job` | `success` | Creating anomaly detection job. | -| `failure` | Failed to create anomaly detection job. | -| `ml_put_ad_datafeed` | `success` | Creating anomaly detection datafeed. | -| `failure` | Failed to create anomaly detection datafeed. | -| `ml_put_calendar` | `success` | Creating calendar. | -| `failure` | Failed to create calendar. | -| `ml_post_calendar_events` | `success` | Adding events to calendar. | -| `failure` | Failed to add events to calendar. | -| `ml_forecast` | `success` | Creating anomaly detection forecast. | -| `failure` | Failed to create anomaly detection forecast. | -| `ml_put_filter` | `success` | Creating filter. | -| `failure` | Failed to create filter. | -| `ml_put_dfa_job` | `success` | Creating data frame analytics job. | -| `failure` | Failed to create data frame analytics job. | -| `ml_put_trained_model` | `success` | Creating trained model. | -| `failure` | Failed to create trained model. | -| `product_documentation_create` | `unknown` | User requested to install the product documentation for use in AI Assistants. | -| `knowledge_base_entry_create` | `success` | User has created knowledge base entry [id=x] | -| `failure` | Failed attempt to create a knowledge base entry | -| `knowledge_base_entry_update` | `success` | User has updated knowledge base entry [id=x] | -| `failure` | Failed attempt to update a knowledge base entry | -| `knowledge_base_entry_delete` | `success` | User has deleted knowledge base entry [id=x] | -| `failure` | Failed attempt to delete a knowledge base entry | -| ##### Type: change [_type_change]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_update` | `unknown` | User is updating a saved object. | -| `failure` | User is not authorized to update a saved object. | -| `saved_object_update_objects_spaces` | `unknown` | User is adding and/or removing a saved object to/from other spaces. | -| `failure` | User is not authorized to add or remove a saved object to or from other spaces. | -| `saved_object_remove_references` | `unknown` | User is removing references to a saved object. | -| `failure` | User is not authorized to remove references to a saved object. | -| `saved_object_collect_multinamespace_references` | `success` | User has accessed references to a multi-space saved object. | -| `failure` | User is not authorized to access references to a multi-space saved object. | -| `connector_update` | `unknown` | User is updating a connector. | -| `failure` | User is not authorized to update a connector. | -| `rule_update` | `unknown` | User is updating a rule. | -| `failure` | User is not authorized to update a rule. | -| `rule_update_api_key` | `unknown` | User is updating the API key of a rule. | -| `failure` | User is not authorized to update the API key of a rule. | -| `rule_enable` | `unknown` | User is enabling a rule. | -| `failure` | User is not authorized to enable a rule. | -| `rule_disable` | `unknown` | User is disabling a rule. | -| `failure` | User is not authorized to disable a rule. | -| `rule_mute` | `unknown` | User is muting a rule. | -| `failure` | User is not authorized to mute a rule. | -| `rule_unmute` | `unknown` | User is unmuting a rule. | -| `failure` | User is not authorized to unmute a rule. | -| `rule_alert_mute` | `unknown` | User is muting an alert. | -| `failure` | User is not authorized to mute an alert. | -| `rule_alert_unmute` | `unknown` | User is unmuting an alert. | -| `failure` | User is not authorized to unmute an alert. | -| `space_update` | `unknown` | User is updating a space. | -| `failure` | User is not authorized to update a space. | -| `alert_update` | `unknown` | User is updating an alert. | -| `failure` | User is not authorized to update an alert. | -| `rule_snooze` | `unknown` | User is snoozing a rule. | -| `failure` | User is not authorized to snooze a rule. | -| `rule_unsnooze` | `unknown` | User is unsnoozing a rule. | -| `failure` | User is not authorized to unsnooze a rule. | -| `case_update` | `unknown` | User is updating a case. | -| `failure` | User is not authorized to update a case. | -| `case_push` | `unknown` | User is pushing a case to an external service. | -| `failure` | User is not authorized to push a case to an external service. | -| `case_configuration_update` | `unknown` | User is updating a case configuration. | -| `failure` | User is not authorized to update a case configuration. | -| `case_comment_update` | `unknown` | User is updating a case comment. | -| `failure` | User is not authorized to update a case comment. | -| `case_user_action_add_case_assignees` | `success` | User has added a case assignee. | -| `case_user_action_update_case_connector` | `success` | User has updated a case connector. | -| `case_user_action_update_case_description` | `success` | User has updated a case description. | -| `case_user_action_update_case_settings` | `success` | User has updated the case settings. | -| `case_user_action_update_case_severity` | `success` | User has updated the case severity. | -| `case_user_action_update_case_status` | `success` | User has updated the case status. | -| `case_user_action_pushed_case` | `success` | User has pushed a case to an external service. | -| `case_user_action_add_case_tags` | `success` | User has added tags to a case. | -| `case_user_action_update_case_title` | `success` | User has updated the case title. | -| `ml_open_ad_job` | `success` | Opening anomaly detection job. | -| `failure` | Failed to open anomaly detection job. | -| `ml_close_ad_job` | `success` | Closing anomaly detection job. | -| `failure` | Failed to close anomaly detection job. | -| `ml_start_ad_datafeed` | `success` | Starting anomaly detection datafeed. | -| `failure` | Failed to start anomaly detection datafeed. | -| `ml_stop_ad_datafeed` | `success` | Stopping anomaly detection datafeed. | -| `failure` | Failed to stop anomaly detection datafeed. | -| `ml_update_ad_job` | `success` | Updating anomaly detection job. | -| `failure` | Failed to update anomaly detection job. | -| `ml_reset_ad_job` | `success` | Resetting anomaly detection job. | -| `failure` | Failed to reset anomaly detection job. | -| `ml_revert_ad_snapshot` | `success` | Reverting anomaly detection snapshot. | -| `failure` | Failed to revert anomaly detection snapshot. | -| `ml_update_ad_datafeed` | `success` | Updating anomaly detection datafeed. | -| `failure` | Failed to update anomaly detection datafeed. | -| `ml_put_calendar_job` | `success` | Adding job to calendar. | -| `failure` | Failed to add job to calendar. | -| `ml_delete_calendar_job` | `success` | Removing job from calendar. | -| `failure` | Failed to remove job from calendar. | -| `ml_update_filter` | `success` | Updating filter. | -| `failure` | Failed to update filter. | -| `ml_start_dfa_job` | `success` | Starting data frame analytics job. | -| `failure` | Failed to start data frame analytics job. | -| `ml_stop_dfa_job` | `success` | Stopping data frame analytics job. | -| `failure` | Failed to stop data frame analytics job. | -| `ml_update_dfa_job` | `success` | Updating data frame analytics job. | -| `failure` | Failed to update data frame analytics job. | -| `ml_start_trained_model_deployment` | `success` | Starting trained model deployment. | -| `failure` | Failed to start trained model deployment. | -| `ml_stop_trained_model_deployment` | `success` | Stopping trained model deployment. | -| `failure` | Failed to stop trained model deployment. | -| `ml_update_trained_model_deployment` | `success` | Updating trained model deployment. | -| `failure` | Failed to update trained model deployment. | -| `product_documentation_update` | `unknown` | User requested to update the product documentation for use in AI Assistants. | -| ##### Type: deletion [_type_deletion]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_delete` | `unknown` | User is deleting a saved object. | -| `failure` | User is not authorized to delete a saved object. | -| `saved_object_close_point_in_time` | `unknown` | User is deleting a Point In Time that was used to query saved objects. | -| `failure` | User is not authorized to delete a Point In Time. | -| `connector_delete` | `unknown` | User is deleting a connector. | -| `failure` | User is not authorized to delete a connector. | -| `rule_delete` | `unknown` | User is deleting a rule. | -| `failure` | User is not authorized to delete a rule. | -| `ad_hoc_run_delete` | `unknown` | User is deleting an ad hoc run. | -| `failure` | User is not authorized to delete an ad hoc run. | -| `space_delete` | `unknown` | User is deleting a space. | -| `failure` | User is not authorized to delete a space. | -| `case_delete` | `unknown` | User is deleting a case. | -| `failure` | User is not authorized to delete a case. | -| `case_comment_delete_all` | `unknown` | User is deleting all comments associated with a case. | -| `failure` | User is not authorized to delete all comments associated with a case. | -| `case_comment_delete` | `unknown` | User is deleting a case comment. | -| `failure` | User is not authorized to delete a case comment. | -| `case_user_action_delete_case_assignees` | `success` | User has removed a case assignee. | -| `case_user_action_delete_comment` | `success` | User has deleted a case comment. | -| `case_user_action_delete_case` | `success` | User has deleted a case. | -| `case_user_action_delete_case_tags` | `success` | User has removed tags from a case. | -| `ml_delete_ad_job` | `success` | Deleting anomaly detection job. | -| `failure` | Failed to delete anomaly detection job. | -| `ml_delete_model_snapshot` | `success` | Deleting model snapshot. | -| `failure` | Failed to delete model snapshot. | -| `ml_delete_ad_datafeed` | `success` | Deleting anomaly detection datafeed. | -| `failure` | Failed to delete anomaly detection datafeed. | -| `ml_delete_calendar` | `success` | Deleting calendar. | -| `failure` | Failed to delete calendar. | -| `ml_delete_calendar_event` | `success` | Deleting calendar event. | -| `failure` | Failed to delete calendar event. | -| `ml_delete_filter` | `success` | Deleting filter. | -| `failure` | Failed to delete filter. | -| `ml_delete_forecast` | `success` | Deleting forecast. | -| `failure` | Failed to delete forecast. | -| `ml_delete_dfa_job` | `success` | Deleting data frame analytics job. | -| `failure` | Failed to delete data frame analytics job. | -| `ml_delete_trained_model` | `success` | Deleting trained model. | -| `failure` | Failed to delete trained model. | -| `product_documentation_delete` | `unknown` | User requested to delete the product documentation for use in AI Assistants. | -| ##### Type: access [_type_access]


| -| **Action** | **Outcome** | **Description** | -| `saved_object_get` | `success` | User has accessed a saved object. | -| `failure` | User is not authorized to access a saved object. | -| `saved_object_resolve` | `success` | User has accessed a saved object. | -| `failure` | User is not authorized to access a saved object. | -| `saved_object_find` | `success` | User has accessed a saved object as part of a search operation. | -| `failure` | User is not authorized to search for saved objects. | -| `connector_get` | `success` | User has accessed a connector. | -| `failure` | User is not authorized to access a connector. | -| `connector_find` | `success` | User has accessed a connector as part of a search operation. | -| `failure` | User is not authorized to search for connectors. | -| `rule_get` | `success` | User has accessed a rule. | -| `failure` | User is not authorized to access a rule. | -| `rule_get_execution_log` | `success` | User has accessed execution log for a rule. | -| `failure` | User is not authorized to access execution log for a rule. | -| `rule_find` | `success` | User has accessed a rule as part of a search operation. | -| `failure` | User is not authorized to search for rules. | -| `rule_schedule_backfill` | `success` | User has accessed a rule as part of a backfill schedule operation. | -| `failure` | User is not authorized to access rule for backfill scheduling. | -| `ad_hoc_run_get` | `success` | User has accessed an ad hoc run. | -| `failure` | User is not authorized to access ad hoc run. | -| `ad_hoc_run_find` | `success` | User has accessed an ad hoc run as part of a search operation. | -| `failure` | User is not authorized to search for ad hoc runs. | -| `space_get` | `success` | User has accessed a space. | -| `failure` | User is not authorized to access a space. | -| `space_find` | `success` | User has accessed a space as part of a search operation. | -| `failure` | User is not authorized to search for spaces. | -| `alert_get` | `success` | User has accessed an alert. | -| `failure` | User is not authorized to access an alert. | -| `alert_find` | `success` | User has accessed an alert as part of a search operation. | -| `failure` | User is not authorized to access alerts. | -| `case_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_bulk_get` | `success` | User has accessed multiple cases. | -| `failure` | User is not authorized to access multiple cases. | -| `case_resolve` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_find` | `success` | User has accessed a case as part of a search operation. | -| `failure` | User is not authorized to search for cases. | -| `case_ids_by_alert_id_get` | `success` | User has accessed cases. | -| `failure` | User is not authorized to access cases. | -| `case_get_metrics` | `success` | User has accessed metrics for a case. | -| `failure` | User is not authorized to access metrics for a case. | -| `cases_get_metrics` | `success` | User has accessed metrics for cases. | -| `failure` | User is not authorized to access metrics for cases. | -| `case_configuration_find` | `success` | User has accessed a case configuration as part of a search operation. | -| `failure` | User is not authorized to search for case configurations. | -| `case_comment_get_metrics` | `success` | User has accessed metrics for case comments. | -| `failure` | User is not authorized to access metrics for case comments. | -| `case_comment_alerts_attach_to_case` | `success` | User has accessed case alerts. | -| `failure` | User is not authorized to access case alerts. | -| `case_comment_get` | `success` | User has accessed a case comment. | -| `failure` | User is not authorized to access a case comment. | -| `case_comment_bulk_get` | `success` | User has accessed multiple case comments. | -| `failure` | User is not authorized to access multiple case comments. | -| `case_comment_get_all` | `success` | User has accessed case comments. | -| `failure` | User is not authorized to access case comments. | -| `case_comment_find` | `success` | User has accessed a case comment as part of a search operation. | -| `failure` | User is not authorized to search for case comments. | -| `case_categories_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_tags_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_reporters_get` | `success` | User has accessed a case. | -| `failure` | User is not authorized to access a case. | -| `case_find_statuses` | `success` | User has accessed a case as part of a search operation. | -| `failure` | User is not authorized to search for cases. | -| `case_user_actions_get` | `success` | User has accessed the user activity of a case. | -| `failure` | User is not authorized to access the user activity of a case. | -| `case_user_actions_find` | `success` | User has accessed the user activity of a case as part of a search operation. | -| `failure` | User is not authorized to access the user activity of a case. | -| `case_user_action_get_metrics` | `success` | User has accessed metrics for the user activity of a case. | -| `failure` | User is not authorized to access metrics for the user activity of a case. | -| `case_user_action_get_users` | `success` | User has accessed the users associated with a case. | -| `failure` | User is not authorized to access the users associated with a case. | -| `case_connectors_get` | `success` | User has accessed the connectors of a case. | -| `failure` | User is not authorized to access the connectors of a case. | -| `ml_infer_trained_model` | `success` | Inferring using trained model. | -| `failure` | Failed to infer using trained model. | -| #### Category: web [_category_web]


| -| **Action** | **Outcome** | **Description** | -| `http_request` | `unknown` | User is making an HTTP request. | - - -## Audit schema [xpack-security-ecs-audit-schema] - -Audit logs are written in JSON using [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/1.6/index.html) specification. - -| | -| --- | -| #### Base Fields [_base_fields]


| -| **Field** | **Description** | -| `@timestamp` | Time when the event was generated.
Example: `2016-05-23T08:05:34.853Z` | -| `message` | Human readable description of the event. | -| #### Event Fields [_event_fields]


| -| **Field** | **Description** | -| $$$field-event-action$$$ `event.action` | The action captured by the event.
Refer to [Audit events](#xpack-security-ecs-audit-logging) for a table of possible actions. | -| $$$field-event-category$$$ `event.category` | High level category associated with the event.
This field is closely related to `event.type`, which is used as a subcategory.
Possible values:`database`,`web`,`authentication` | -| $$$field-event-type$$$ `event.type` | Subcategory associated with the event.
This field can be used along with the `event.category` field to enable filtering events down to a level appropriate for single visualization.
Possible values:`creation`,`access`,`change`,`deletion` | -| $$$field-event-outcome$$$ `event.outcome` | Denotes whether the event represents a success or failure:

* Any actions that the user is not authorized to perform are logged with outcome: `failure`
* Authorized read operations are only logged after successfully fetching the data from {{es}} with outcome: `success`
* Authorized create, update, or delete operations are logged before attempting the operation in {{es}} with outcome: `unknown`

Possible values: `success`, `failure`, `unknown`
| -| #### User Fields [_user_fields]


| -| **Field** | **Description** | -| `user.id` | Unique identifier of the user across sessions (See [user profiles](../../users-roles/cluster-or-deployment-auth/user-profiles.md)). | -| `user.name` | Login name of the user.
Example: `jdoe` | -| `user.roles[]` | Set of user roles at the time of the event.
Example: `[kibana_admin, reporting_user]` | -| #### Kibana Fields [_kibana_fields]


| -| **Field** | **Description** | -| `kibana.space_id` | ID of the space associated with the event.
Example: `default` | -| `kibana.session_id` | ID of the user session associated with the event.
Each login attempt results in a unique session id. | -| `kibana.saved_object.type` | Type of saved object associated with the event.
Example: `dashboard` | -| `kibana.saved_object.id` | ID of the saved object associated with the event. | -| `kibana.authentication_provider` | Name of the authentication provider associated with the event.
Example: `my-saml-provider` | -| `kibana.authentication_type` | Type of the authentication provider associated with the event.
Example: `saml` | -| `kibana.authentication_realm` | Name of the Elasticsearch realm that has authenticated the user.
Example: `native` | -| `kibana.lookup_realm` | Name of the Elasticsearch realm where the user details were retrieved from.
Example: `native` | -| `kibana.add_to_spaces[]` | Set of space IDs that a saved object is being shared to as part of the event.
Example: `[default, marketing]` | -| `kibana.delete_from_spaces[]` | Set of space IDs that a saved object is being removed from as part of the event.
Example: `[marketing]` | -| #### Error Fields [_error_fields]


| -| **Field** | **Description** | -| `error.code` | Error code describing the error. | -| `error.message` | Error message. | -| #### HTTP and URL Fields [_http_and_url_fields]


| -| **Field** | **Description** | -| `client.ip` | Client IP address. | -| `http.request.method` | HTTP request method.
Example: `get`, `post`, `put`, `delete` | -| `http.request.headers.x-forwarded-for` | `X-Forwarded-For` request header used to identify the originating client IP address when connecting through proxy servers.
Example: `161.66.20.177, 236.198.214.101` | -| `url.domain` | Domain of the URL.
Example: `www.elastic.co` | -| `url.path` | Path of the request.
Example: `/search` | -| `url.port` | Port of the request.
Example: `443` | -| `url.query` | The query field describes the query string of the request.
Example: `q=elasticsearch` | -| `url.scheme` | Scheme of the request.
Example: `https` | -| #### Tracing Fields [_tracing_fields]


| -| **Field** | **Description** | -| $$$field-trace-id$$$ `trace.id` | Unique identifier allowing events of the same transaction from {{kib}} and {{es}} to be correlated. | - - -## Correlating audit events [xpack-security-ecs-audit-correlation] - -Audit events can be correlated in two ways: - -1. Multiple {{kib}} audit events that resulted from the same request can be correlated together. -2. If [{{es}} audit logging](enabling-elasticsearch-audit-logs.md) is enabled, {{kib}} audit events from one request can be correlated with backend calls that create {{es}} audit events. - -::::{note} -The examples below are simplified, many fields have been omitted and values have been shortened for clarity. -:::: - - -### Example 1: correlating multiple {{kib}} audit events [_example_1_correlating_multiple_kib_audit_events] - -When "thom" creates a new alerting rule, five audit events are written: - -```json -{"event":{"action":"http_request","category":["web"],"outcome":"unknown"},"http":{"request":{"method":"post"}},"url":{"domain":"localhost","path":"/api/alerting/rule","port":5601,"scheme":"https"},"user":{"name":"thom","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"3dHCZRB..."},"@timestamp":"2022-01-25T13:05:34.449-05:00","message":"User is requesting [/api/alerting/rule] endpoint","trace":{"id":"e300e06..."}} -{"event":{"action":"space_get","category":["database"],"type":["access"],"outcome":"success"},"kibana":{"space_id":"default","session_id":"3dHCZRB...","saved_object":{"type":"space","id":"default"}},"user":{"name":"thom","roles":["superuser"]},"@timestamp":"2022-01-25T13:05:34.454-05:00","message":"User has accessed space [id=default]","trace":{"id":"e300e06..."}} -{"event":{"action":"connector_get","category":["database"],"type":["access"],"outcome":"success"},"kibana":{"space_id":"default","session_id":"3dHCZRB...","saved_object":{"type":"action","id":"5e3b1ae..."}},"user":{"name":"thom","roles":["superuser"]},"@timestamp":"2022-01-25T13:05:34.948-05:00","message":"User has accessed connector [id=5e3b1ae...]","trace":{"id":"e300e06..."}} -{"event":{"action":"connector_get","category":["database"],"type":["access"],"outcome":"success"},"kibana":{"space_id":"default","session_id":"3dHCZRB...","saved_object":{"type":"action","id":"5e3b1ae..."}},"user":{"name":"thom","roles":["superuser"]},"@timestamp":"2022-01-25T13:05:34.956-05:00","message":"User has accessed connector [id=5e3b1ae...]","trace":{"id":"e300e06..."}} -{"event":{"action":"rule_create","category":["database"],"type":["creation"],"outcome":"unknown"},"kibana":{"space_id":"default","session_id":"3dHCZRB...","saved_object":{"type":"alert","id":"64517c3..."}},"user":{"name":"thom","roles":["superuser"]},"@timestamp":"2022-01-25T13:05:34.956-05:00","message":"User is creating rule [id=64517c3...]","trace":{"id":"e300e06..."}} -``` - -All of these audit events can be correlated together by the same `trace.id` value `"e300e06..."`. The first event is the HTTP API call, the next audit events are checks to validate the space and the connectors, and the last audit event is the actual rule creation. - - -### Example 2: correlating a {{kib}} audit event with {{es}} audit events [_example_2_correlating_a_kib_audit_event_with_es_audit_events] - -When "thom" logs in, a "user_login" {{kib}} audit event is written: - -```json -{"event":{"action":"user_login","category":["authentication"],"outcome":"success"},"kibana":{"session_id":"ab93zdA..."},"user":{"name":"thom","roles":["superuser"]},"@timestamp":"2022-01-25T09:40:39.267-05:00","message":"User [thom] has logged in using basic provider [name=basic]","trace":{"id":"818cbf3..."}} -``` - -The `trace.id` value `"818cbf3..."` in the {{kib}} audit event can be correlated with the `opaque_id` value in these six {{es}} audit events: - -```json -{"type":"audit", "timestamp":"2022-01-25T09:40:38,604-0500", "event.action":"access_granted", "user.name":"thom", "user.roles":["superuser"], "request.id":"YCx8wxs...", "action":"cluster:admin/xpack/security/user/authenticate", "request.name":"AuthenticateRequest", "opaque_id":"818cbf3..."} -{"type":"audit", "timestamp":"2022-01-25T09:40:38,613-0500", "event.action":"access_granted", "user.name":"kibana_system", "user.roles":["kibana_system"], "request.id":"Ksx73Ad...", "action":"indices:data/write/index", "request.name":"IndexRequest", "indices":[".kibana_security_session_1"], "opaque_id":"818cbf3..."} -{"type":"audit", "timestamp":"2022-01-25T09:40:38,613-0500", "event.action":"access_granted", "user.name":"kibana_system", "user.roles":["kibana_system"], "request.id":"Ksx73Ad...", "action":"indices:data/write/bulk", "request.name":"BulkRequest", "opaque_id":"818cbf3..."} -{"type":"audit", "timestamp":"2022-01-25T09:40:38,613-0500", "event.action":"access_granted", "user.name":"kibana_system", "user.roles":["kibana_system"], "request.id":"Ksx73Ad...", "action":"indices:data/write/bulk[s]", "request.name":"BulkShardRequest", "indices":[".kibana_security_session_1"], "opaque_id":"818cbf3..."} -{"type":"audit", "timestamp":"2022-01-25T09:40:38,613-0500", "event.action":"access_granted", "user.name":"kibana_system", "user.roles":["kibana_system"], "request.id":"Ksx73Ad...", "action":"indices:data/write/index:op_type/create", "request.name":"BulkItemRequest", "indices":[".kibana_security_session_1"], "opaque_id":"818cbf3..."} -{"type":"audit", "timestamp":"2022-01-25T09:40:38,613-0500", "event.action":"access_granted", "user.name":"kibana_system", "user.roles":["kibana_system"], "request.id":"Ksx73Ad...", "action":"indices:data/write/bulk[s][p]", "request.name":"BulkShardRequest", "indices":[".kibana_security_session_1"], "opaque_id":"818cbf3..."} -``` - -The {{es}} audit events show that "thom" authenticated, then subsequently "kibana_system" created a session for that user. diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 7d12c21f70..27b2873b6a 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -8,7 +8,27 @@ applies: # Security event audit logging % What needs to be done: Write from scratch - % GitHub issue: https://github.com/elastic/docs-projects/issues/350 +% Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. + +::::{important} +Audit logs are only available on certain [subscription levels](https://www.elastic.co/subscriptions). +:::: + +Audit logging is a powerful feature that helps you monitor and track security-related events within the {{stack}}. By enabling audit logs, you can gain visibility into authentication attempts, authorization decisions, and other system activity. + +This section provides a comprehensive guide to configuring and using audit logging in Elasticsearch and Kibana across all supported deployment types, including self-managed clusters, Elastic Cloud, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). + +In this section you will learn how to: + +* Enable {{es}} audit logs for all supported deployment types. + +* Enable {{kib}} audit logs to track authentication, API access, and administrative actions. + +* Configure {{es}} audit logging framework by using [include/exclude filters](elasticsearch-audit-events.md), [ignore policies](logfile-audit-events-ignore-policies.md), and [search queries auditing](auditing-search-queries.md). + +* Correlating {{kib}} and {{es}} audit events to gain a complete view of security-related activities. + +By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. -% Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. \ No newline at end of file +Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. \ No newline at end of file diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 6102870923..c8b377e4c6 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -779,14 +779,12 @@ toc: - file: monitor/logging-configuration/kibana-logging-cli-configuration.md - file: monitor/logging-configuration/security-event-audit-logging.md children: - - file: monitor/logging-configuration/enabling-elasticsearch-audit-logs.md + - file: monitor/logging-configuration/enabling-audit-logs.md children: - file: monitor/logging-configuration/elasticsearch-audit-events.md - file: monitor/logging-configuration/logfile-audit-output.md - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - file: monitor/logging-configuration/auditing-search-queries.md - - file: monitor/logging-configuration/enabling-kibana-audit-logs.md - - file: monitor/logging-configuration/enabling-audit-logs-in-orchestrated-deployments.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md - file: cloud-organization.md children: diff --git a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md b/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md deleted file mode 100644 index 59dcab17b6..0000000000 --- a/raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md +++ /dev/null @@ -1,40 +0,0 @@ -# Audit logging [k8s_audit_logging] - -Audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section when audit logging is enabled (it is disabled by default). - -```yaml -apiVersion: elasticsearch.k8s.elastic.co/v1 -kind: Elasticsearch -spec: - monitoring: - metrics: - elasticsearchRefs: - - name: monitoring - namespace: observability - logs: - elasticsearchRefs: - - name: monitoring - namespace: observability - nodeSets: - - name: default - config: - # https://www.elastic.co/guide/en/elasticsearch/reference/current/enable-audit-logging.html - xpack.security.audit.enabled: true ---- -apiVersion: kibana.k8s.elastic.co/v1 -kind: Kibana -spec: - monitoring: - metrics: - elasticsearchRefs: - - name: monitoring - namespace: observability - logs: - elasticsearchRefs: - - name: monitoring - namespace: observability - config: - # https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html - xpack.security.audit.enabled: true -``` - diff --git a/raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md b/raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md deleted file mode 100644 index 7d5fda5314..0000000000 --- a/raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md +++ /dev/null @@ -1,33 +0,0 @@ -# Enable auditing [ece-enable-auditing] - -With auditing enabled you can keep track of security-related events, such as successful and unsuccessful authorization attempts on the cluster. In Elastic Cloud Enterprise, to get audit events for both Elasticsearch and Kibana, you need to enable auditing for each component separately. - -To enable auditing for Elasticsearch: - -1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From your deployment menu, go to the **Edit** page. -4. In the **Elasticsearch** section, select **Edit user settings and plugins**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for the first node instead. -5. Add the setting `xpack.security.audit.enabled: true`. -6. Select **Save**. - -For more information and other available auditing settings in Elasticsearch, check [Auditing security settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). Audit logs can be viewed within Elasticsearch logs. - -To enable auditing for Kibana: - -1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. - - Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters. - -3. From your deployment menu, go to the **Edit** page. -4. In the **Kibana** section, select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. -5. Add the setting `xpack.security.audit.enabled: true`. -6. If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. -7. Select **Save**. - -For more information about audit logging in Kibana, check [Audit Logging](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html). - diff --git a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md index f30761a883..2c2a378284 100644 --- a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md +++ b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md @@ -224,6 +224,7 @@ With logging and monitoring enabled for a deployment, metrics are collected for #### Enabling Elasticsearch/Kibana audit logs on your deployment [ec-enable-audit-logs] +% Added by eedugon to audit logging in deploy and manage -> monitoring -> logging section Audit logs are useful for tracking security events on your {{es}} and/or {{kib}} clusters. To enable {{es}} audit logs on your deployment: diff --git a/raw-migrated-files/toc.yml b/raw-migrated-files/toc.yml index ab854a0347..5d9e2fbe78 100644 --- a/raw-migrated-files/toc.yml +++ b/raw-migrated-files/toc.yml @@ -13,7 +13,6 @@ toc: - file: apm-agent-ruby/apm-agent-ruby/release-notes-4.x.md - file: cloud-on-k8s/cloud-on-k8s/index.md children: - - file: cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md - file: cloud-on-k8s/cloud-on-k8s/k8s_installation.md - file: cloud-on-k8s/cloud-on-k8s/k8s_learn_more_about_eck.md - file: cloud-on-k8s/cloud-on-k8s/k8s-accessing-elastic-services.md @@ -56,7 +55,6 @@ toc: - file: cloud/cloud-enterprise/ece-delete-deployment.md - file: cloud/cloud-enterprise/ece-deployment-no-op.md - file: cloud/cloud-enterprise/ece-disable-data-tier.md - - file: cloud/cloud-enterprise/ece-enable-auditing.md - file: cloud/cloud-enterprise/ece-find.md - file: cloud/cloud-enterprise/ece-generate-roles-token.md - file: cloud/cloud-enterprise/ece-getting-started-node-js.md From d870029679c15153600b9ffc62066c82853cc4fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 5 Feb 2025 16:38:54 +0100 Subject: [PATCH 02/27] elasticsearch audit events prepared to be moved to reference --- .../elasticsearch-audit-events.md | 24 +++++++++++++++---- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md index 7bcf440eed..0c01369dfc 100644 --- a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md +++ b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md @@ -8,18 +8,32 @@ applies: stack: all --- -% Part of this doc should probably go to reference content at a later stage +% Pending to move this page to REFERENCE -# Elasticsearch audit events [audit-event-types] +# Elasticsearch audit events [elasticsearch-audit-events] -When you are [auditing security events](enabling-audit-logs.md), a single client request might generate multiple audit events, across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. +When you are [auditing security events](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md), a single client request might generate multiple audit events, across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. -Use the [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) setting in `elasticsearch.yml` to specify the kind of events you want to include in the auditing output. +This document provides a reference for all audit event types and their associated attributes in {{es}}. These types and attributes can be used to filter audit logs using inclusion and exclusion rules or to define [ignore policies](/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md). ::::{note} -Certain audit events require the `security_config_change` event type to audit the related event action. The description of impacted audit events indicate whether that event type is required. +Certain audit events require the `security_config_change` event type to log the related event action. The event descriptions in this document indicate whether this requirement is applicable. :::: +# Elasticsearch Audit Events [elasticsearch-audit-events] + +When [auditing security events](enabling-audit-logs.md), a single client request may generate multiple audit events across different cluster nodes. The `request.id` attribute is a common identifier that can be used to correlate related events. + + +To control which events are logged, use the [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) setting in `elasticsearch.yml` to specify the event types to include in the audit output. + +::::{note} +Some audit events require the `security_config_change` event type to log specific event actions. The event descriptions in this document indicate when this requirement applies. +Some audit events require the `security_config_change` event type to log specific event actions. The event descriptions in this document indicate when this requirement applies. +:::: + + +## Audit event types [audit-event-types] $$$event-access-denied$$$ From 07e5dec2a36893a91def1cf788d077624d2df0d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 5 Feb 2025 22:35:58 +0100 Subject: [PATCH 03/27] work in progress, applying shaina's suggestions --- .../auditing-search-queries.md | 1 - ...elating-kibana-elasticsearch-audit-logs.md | 3 +- .../elasticsearch-audit-events.md | 17 +-- .../enabling-audit-logs.md | 129 ++++++++++++------ .../logfile-audit-events-ignore-policies.md | 3 +- .../logfile-audit-output.md | 5 +- .../security-event-audit-logging.md | 20 ++- deploy-manage/toc.yml | 6 +- 8 files changed, 114 insertions(+), 70 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index f91abe8fb4..22fdde9364 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -24,7 +24,6 @@ xpack.security.audit.logfile.events.emit_request_body: true No filtering is performed when auditing, so sensitive data might be audited in plain text when audit events include the request body. Also, the request body can contain malicious content that can break a parser consuming the audit logs. :::: - The request body is printed as an escaped JSON string value (RFC 4627) to the `request.body` event attribute. Not all events contain the `request.body` attribute, even when the above setting is toggled. The ones that do are: `authentication_success`, `authentication_failed`, `realm_authentication_failed`, `tampered_request`, `run_as_denied`, and `anonymous_access_denied`. The `request.body` attribute is printed on the coordinating node only (the node that handles the REST request). Most of these event types are [not included by default](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include). diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index dff13687a1..f2b72b4b95 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -1,4 +1,5 @@ --- +navigation_title: Correlate audit events mapped_pages: - https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html applies: @@ -13,7 +14,7 @@ applies: Audit events can be correlated in two ways: 1. Multiple {{kib}} audit events that resulted from the same request can be correlated together. -2. If [{{es}} audit logging](enabling-elasticsearch-audit-logs.md) is enabled, {{kib}} audit events from one request can be correlated with backend calls that create {{es}} audit events. +2. If [{{es}} audit logging](enabling-audit-logs.md) is enabled, {{kib}} audit events from one request can be correlated with backend calls that create {{es}} audit events. ::::{note} The examples below are simplified, many fields have been omitted and values have been shortened for clarity. diff --git a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md index 0c01369dfc..a56f0cae9e 100644 --- a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md +++ b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md @@ -12,27 +12,14 @@ applies: # Elasticsearch audit events [elasticsearch-audit-events] -When you are [auditing security events](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md), a single client request might generate multiple audit events, across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. +When you are [auditing security events](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md), a single client request might generate multiple audit events across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. -This document provides a reference for all audit event types and their associated attributes in {{es}}. These types and attributes can be used to filter audit logs using inclusion and exclusion rules or to define [ignore policies](/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md). +This document provides a reference for all audit event [types](#audit-event-types) and their associated [attributes](#audit-event-attributes) in {{es}}. Types and attributes can be used to control what gets logged through [audit event settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) and [ignore policies](/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md). ::::{note} Certain audit events require the `security_config_change` event type to log the related event action. The event descriptions in this document indicate whether this requirement is applicable. :::: -# Elasticsearch Audit Events [elasticsearch-audit-events] - -When [auditing security events](enabling-audit-logs.md), a single client request may generate multiple audit events across different cluster nodes. The `request.id` attribute is a common identifier that can be used to correlate related events. - - -To control which events are logged, use the [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) setting in `elasticsearch.yml` to specify the event types to include in the audit output. - -::::{note} -Some audit events require the `security_config_change` event type to log specific event actions. The event descriptions in this document indicate when this requirement applies. -Some audit events require the `security_config_change` event type to log specific event actions. The event descriptions in this document indicate when this requirement applies. -:::: - - ## Audit event types [audit-event-types] $$$event-access-denied$$$ diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 82a4305a62..50e0ba046c 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -10,6 +10,7 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- % What needs to be done: Refine @@ -24,31 +25,47 @@ applies: % - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md % - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md +% https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings +% https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings + + # Enable audit logging [enable-audit-logging] You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). -Audit logging also provides forensic evidence in the event of an attack. +This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all deployment types. + +In all deployment types, the auditing settings to enable and configure audit logs are the same. + +As with self-managed systems, audit logging must be enabled separately for both {{es}} and {{kib}} to capture events from both components. + +::::{important} +In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). +:::: ::::{important} Audit logs are **disabled** by default. You must explicitly enable audit logging. :::: -## Enable audit logging in Elasticsearch [enable-audit-logging-elasticsearch] + + + +## Enable audit logging in self-managed clusters [enable-audit-logging-self-managed] +:::{applies} +:stack: all +::: -To enable audit logging: +To enable audit logging in {{es}}: 1. Set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. 2. Restart {{es}}. -When audit logging is enabled, [security events](elasticsearch-audit-events.md) are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, see [Logfile audit output](logfile-audit-output.md). +When audit logging is enabled, audit events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). ::::{note} You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). :::: -## Enable audit logging in Kibana [enable-audit-logging-kibana] - To enable audit logging in {{kib}}: 1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml` @@ -59,18 +76,23 @@ You can optionally configure audit logs location, file/rolling file appenders an :::: ## Enable audit logging in orchestrated deployments [enable-audit-logging-orchestrated] +:::{applies} +:ece: all +:eck: all +:hosted: all +::: -In orchestrated systems you just need to set the relevant setting through the orchestrator, as with any other {{es}} or {{kib}} setting. +To enable {{es}} or {{kib}} audit logs in an orchestrated system, set `xpack.security.audit.enabled` to `true` in the respective {{es}} or {{kib}} configuration through the orchestrator. -Below you have some examples: +If you’re unfamiliar with configuring {{es}} and {{kib}} settings in your orchestrator, use the following tabs for detailed instructions on enabling audit logging for your deployment type. ::::::{tab-set} :::::{tab-item} Elastic Cloud Hosted - -To enable audit logs on your deployment: +To enable audit logging in an Elastic Cloud Hosted deployment: 1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). + 2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. ::::{tip} @@ -78,51 +100,52 @@ To enable audit logs on your deployment: :::: 3. From your deployment menu, go to the **Edit** page. -4. To enable audit logs in {{es}}, in the **Elasticsearch** section select **Manage user settings and extensions**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for each node instead. -5. To enable audit logs in {{kib}}, in the **Kibana** section select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. -6. Add `xpack.security.audit.enabled: true` to the user settings. -7. Select **Save changes**. -A plan change will run on your deployment. When it finishes, audit logs will be delivered to your monitoring deployment. +4. To enable auditing for Elasticsearch: + * In the **Elasticsearch** section select **Manage user settings and extensions**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for each node instead. + * Add the setting `xpack.security.audit.enabled: true`. + +5. To enable auditing for Kibana: + * In the **Kibana** section, select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. + * Add the setting `xpack.security.audit.enabled: true`. + +6. Select **Save changes**. + A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. +For details on available audit settings in Elastic Cloud Hosted deployments, refer to the following documents: +* [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) +* [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) ::::: :::::{tab-item} ECE -In Elastic Cloud Enterprise, to get audit events for both Elasticsearch and Kibana, you need to enable auditing for each component separately. - -To enable auditing for Elasticsearch: +To enable audit logging in an ECE deployment: 1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). + 2. On the **Deployments** page, select your deployment. ::::{tip} On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. :::: + 3. From your deployment menu, go to the **Edit** page. -4. In the **Elasticsearch** section, select **Edit user settings and plugins**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for the first node instead. -5. Add the setting `xpack.security.audit.enabled: true`. -6. Select **Save**. -For more information and other available auditing settings in Elasticsearch, check [Auditing security settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). Audit logs can be viewed within Elasticsearch logs. +4. To enable auditing for {{es}}: + * In the **Elasticsearch** section, select **Edit user settings and plugins**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for the first node instead. + * Add the setting `xpack.security.audit.enabled: true`. -To enable auditing for Kibana: +5. To enable auditing for {{kib}}: + * In the **Kibana** section, select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. + * Add the setting `xpack.security.audit.enabled: true`. + * If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. -1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). -2. On the **Deployments** page, select your deployment. - ::::{tip} - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - :::: -3. From your deployment menu, go to the **Edit** page. -4. In the **Kibana** section, select **Edit user settings**. For deployments with existing user settings, you may have to expand the **Edit kibana.yml** caret instead. -5. Add the setting `xpack.security.audit.enabled: true`. -6. If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. -7. Select **Save**. +6. Select **Save**. + A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. -For more information about audit logging in Kibana, check [Audit Logging](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html). +For more information and other available auditing settings, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html) and [{{kib}} auditing settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). Audit logs can be viewed within Elasticsearch and Kibana logs. ::::: :::::{tab-item} ECK - -Audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section when audit logging is enabled (it is disabled by default). +To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification, for example: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 @@ -160,12 +183,40 @@ spec: xpack.security.audit.enabled: true ``` +When enabled, audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section. If monitoring is not enabled audit logs will only be visible at container level. ::::: :::::: +## Advanced Configuration [audit-logging-configuration] + +Kibana: +You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). + + +Elasticsearch: +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). + + +Consider the following recommendations when working with {{es}} and {{kib}} audit logging: + +(you configure in the same way you have enabled the feature...) + +(reading security events... in the overview) + +(move this)* When you are auditing security events, a single client request might generate multiple audit events across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. +* Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the kind of events you want to include or exclude in the {{es}} auditing output. +* Refer to [logfile ignore policies](./logfile-audit-events-ignore-policies.md) for an example of how to tune the verbosity of the {{es}} audit trail. +* Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. +* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) to determine which events to exclude from the {{kib}} audit log. +* For a complete list of event kinds and attributes, refer to: + * [{{es}} audit events](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). + * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) + +Audit logs generated in orchestrated deployments are similar to those in self-managed clusters, and you can also control which security events to log using all the mechanisms described in [](#audit-logging-configuration). However, in orchestrated environments, changing log file names or `log4j2.properties` settings is not supported. + +For details on available audit settings in Elastic Cloud Hosted deployments, refer to the following documents: +* [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) +* [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) -Orchestrated deployments allow to change certain settings regarding audit logging, such as: -* asdad -* bsdsda diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md index f7a164d4e4..5c8cfbd317 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md @@ -1,4 +1,5 @@ --- +navigation_title: Elasticsearch audit events ignore policies mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/audit-log-ignore-policy.html applies: @@ -8,7 +9,7 @@ applies: stack: all --- -# Logfile audit events ignore policies [audit-log-ignore-policy] +# Elasticsearch logfile audit events ignore policies [audit-log-ignore-policy] The comprehensive audit trail is necessary to ensure accountability. It offers tremendous value during incident response and can even be required for demonstrating compliance. diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index 6ee7bbf238..50e68e1aca 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -1,4 +1,5 @@ --- +navigation_title: Elasticsearch audit output mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/audit-log-output.html applies: @@ -8,9 +9,7 @@ applies: stack: all --- -% evaluate the applies section - -# Logfile audit output [audit-log-output] +# Elasticsearch logfile audit output [audit-log-output] The `logfile` audit output is the only output for auditing. It writes data to the `_audit.json` file in the logs directory. diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 27b2873b6a..c0d08df8c3 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -11,24 +11,30 @@ applies: % GitHub issue: https://github.com/elastic/docs-projects/issues/350 % Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. +% consider organizing the architecture in a better way, separating configuration pages from visualizing pages if needed... + ::::{important} Audit logs are only available on certain [subscription levels](https://www.elastic.co/subscriptions). :::: Audit logging is a powerful feature that helps you monitor and track security-related events within the {{stack}}. By enabling audit logs, you can gain visibility into authentication attempts, authorization decisions, and other system activity. -This section provides a comprehensive guide to configuring and using audit logging in Elasticsearch and Kibana across all supported deployment types, including self-managed clusters, Elastic Cloud, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). +Audit logging also provides forensic evidence in the event of an attack. + +Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. + +This section provides a comprehensive guide to configure and use audit logging in {{es}} and {{kib}} across all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). In this section you will learn how to: -* Enable {{es}} audit logs for all supported deployment types. +* [Enable {{es}} audit logs](./enabling-audit-logs.md#enable-audit-logging-elasticsearch). -* Enable {{kib}} audit logs to track authentication, API access, and administrative actions. +* [Enable {{kib}} audit logs](./enabling-audit-logs.md#enable-audit-logging-kibana). -* Configure {{es}} audit logging framework by using [include/exclude filters](elasticsearch-audit-events.md), [ignore policies](logfile-audit-events-ignore-policies.md), and [search queries auditing](auditing-search-queries.md). +* [Enable audit logs on orchestrated deployments](./enabling-audit-logs.md#enable-audit-logging-orchestrated). -* Correlating {{kib}} and {{es}} audit events to gain a complete view of security-related activities. +* [Configure audit logging](./enabling-audit-logs.md#audit-logging-configuration) to control the events that should be logged. -By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. +* [Correlate {{kib}} and {{es}} audit events](./correlating-kibana-elasticsearch-audit-logs.md) to gain a complete view of security-related activities. -Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. \ No newline at end of file +By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. \ No newline at end of file diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index c8b377e4c6..52f186c644 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -782,9 +782,9 @@ toc: - file: monitor/logging-configuration/enabling-audit-logs.md children: - file: monitor/logging-configuration/elasticsearch-audit-events.md - - file: monitor/logging-configuration/logfile-audit-output.md - - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - - file: monitor/logging-configuration/auditing-search-queries.md + - file: monitor/logging-configuration/logfile-audit-output.md + - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md + - file: monitor/logging-configuration/auditing-search-queries.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md - file: cloud-organization.md children: From 460db8e275ce253384e75335fb424f51f3ffb936 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 13:04:53 +0100 Subject: [PATCH 04/27] audit logging updated --- .../auditing-search-queries.md | 6 +- .../enabling-audit-logs.md | 103 +++++++++++------- .../logfile-audit-output.md | 23 ++-- .../security-event-audit-logging.md | 35 ++++-- 4 files changed, 98 insertions(+), 69 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index 22fdde9364..16907b747d 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -14,14 +14,16 @@ There is no [audit event type](elasticsearch-audit-events.md) specifically dedic Search queries are contained inside HTTP request bodies, however, and some audit events that are generated by the REST layer, on the coordinating node, can be toggled to output the request body to the audit log. Therefore, one must audit request bodies in order to audit search queries. -To make certain audit events include the request body, edit the following setting in the `elasticsearch.yml` file: +To make certain audit events include the request body, configure the following setting in {{es}}: ```yaml xpack.security.audit.logfile.events.emit_request_body: true ``` +In self-managed deployments, add the to the `elasticsearch.yml` file, while in orchestrated environments, apply the setting through the appropriate [configuration mechanism](/deploy-manage/deploy.md). + ::::{important} -No filtering is performed when auditing, so sensitive data might be audited in plain text when audit events include the request body. Also, the request body can contain malicious content that can break a parser consuming the audit logs. +No filtering is performed when auditing, so **sensitive data might be audited in plain text when audit events include the request body**. Also, the request body can contain malicious content that can break a parser consuming the audit logs. :::: The request body is printed as an escaped JSON string value (RFC 4627) to the `request.body` event attribute. diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 50e0ba046c..09add39039 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -28,32 +28,33 @@ applies: % https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings % https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings - # Enable audit logging [enable-audit-logging] +::::{important} +Audit logs are **disabled** by default and must be explicitly enabled; they are available only on certain [subscription levels](https://www.elastic.co/subscriptions). +:::: + You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). -This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all deployment types. +This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -In all deployment types, the auditing settings to enable and configure audit logs are the same. +Audit logging is enabled separately for both {{es}} and {{kib}}, and enabling both is optional, depending on your use case and requirements. -As with self-managed systems, audit logging must be enabled separately for both {{es}} and {{kib}} to capture events from both components. +The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. -::::{important} -In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). -:::: +The only difference lies in how the configuration is applied: + * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files. + * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. -::::{important} -Audit logs are **disabled** by default. You must explicitly enable audit logging. -:::: +In short, to enable {{es}} or {{kib}} audit logs, set `xpack.security.audit.enabled` to `true` in the configuration of **all {{es}} or {{kib}} nodes**. In self-managed clusters you will have to perform a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) to apply the changes, while in orchestrated environments the orchestrator will apply the change and restart the nodes automatically. + +The following tabs show the detailed process for all deployment types: - - +## Enabling procedure [enable-audit-logging-procedure] -## Enable audit logging in self-managed clusters [enable-audit-logging-self-managed] -:::{applies} -:stack: all -::: +::::::{tab-set} + +:::::{tab-item} Self-managed To enable audit logging in {{es}}: @@ -75,20 +76,14 @@ To enable audit logging in {{kib}}: You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). :::: -## Enable audit logging in orchestrated deployments [enable-audit-logging-orchestrated] -:::{applies} -:ece: all -:eck: all -:hosted: all -::: - -To enable {{es}} or {{kib}} audit logs in an orchestrated system, set `xpack.security.audit.enabled` to `true` in the respective {{es}} or {{kib}} configuration through the orchestrator. +::::: -If you’re unfamiliar with configuring {{es}} and {{kib}} settings in your orchestrator, use the following tabs for detailed instructions on enabling audit logging for your deployment type. +:::::{tab-item} Elastic Cloud Hosted -::::::{tab-set} +::::{important} +In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). +:::: -:::::{tab-item} Elastic Cloud Hosted To enable audit logging in an Elastic Cloud Hosted deployment: 1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). @@ -118,6 +113,11 @@ For details on available audit settings in Elastic Cloud Hosted deployments, ref ::::: :::::{tab-item} ECE + +::::{important} +In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). +:::: + To enable audit logging in an ECE deployment: 1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). @@ -145,6 +145,11 @@ For more information and other available auditing settings, refer to [{{es}} aud ::::: :::::{tab-item} ECK + +::::{important} +In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). +:::: + To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification, for example: ```yaml @@ -190,33 +195,47 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref ## Advanced Configuration [audit-logging-configuration] -Kibana: -You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). - +{{es}} audit logging and {{kib}} audit logging offer several configuration mechanisms to control what events are logged and what information is included in the audit log. For more information, refer to: -Elasticsearch: -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). +* [{{es}} audited event settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) +* [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) +* [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) +* [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings) +::::{important} +Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. +:::: -Consider the following recommendations when working with {{es}} and {{kib}} audit logging: - -(you configure in the same way you have enabled the feature...) - -(reading security events... in the overview) +The configuration can be applied using the same [procedure](#enable-audit-logging-procedure) used to enable audit logs, described for all deployment types. -(move this)* When you are auditing security events, a single client request might generate multiple audit events across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. +Configuration suggestions: * Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the kind of events you want to include or exclude in the {{es}} auditing output. * Refer to [logfile ignore policies](./logfile-audit-events-ignore-policies.md) for an example of how to tune the verbosity of the {{es}} audit trail. * Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. * Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) to determine which events to exclude from the {{kib}} audit log. + +Extra considerations: + +* Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: + * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) + * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) + +* Changing the configuration of `log4j2.properties` is not supported in orchestrated deployments, and it's not recommended for `self-managed` deployments. + * For a complete list of event kinds and attributes, refer to: * [{{es}} audit events](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) -Audit logs generated in orchestrated deployments are similar to those in self-managed clusters, and you can also control which security events to log using all the mechanisms described in [](#audit-logging-configuration). However, in orchestrated environments, changing log file names or `log4j2.properties` settings is not supported. -For details on available audit settings in Elastic Cloud Hosted deployments, refer to the following documents: -* [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) -* [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) +## Elasticsearch audit output + + + +You can also configure how the logfile is written in the `log4j2.properties` file located in `ES_PATH_CONF` (or check out the relevant portion of the [log4j2.properties in the sources](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/config/log4j2.properties)). By default, audit information is appended to the `_audit.json` file located in the standard Elasticsearch `logs` directory (typically located at `$ES_HOME/logs`). The file is also rotated and archived daily or upon reaching the 1GB file size limit. + + + + + diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index 50e68e1aca..5474ff18c0 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -11,13 +11,17 @@ applies: # Elasticsearch logfile audit output [audit-log-output] -The `logfile` audit output is the only output for auditing. It writes data to the `_audit.json` file in the logs directory. +The `logfile` audit output is the only output for auditing. By default, it writes data to the `_audit.json` file in the logs directory. The file is also rotated and archived daily or upon reaching the 1GB file size limit. + +In self-managed clusters, you can configure how the `logfile` is written in the `log4j2.properties` file located in `ES_PATH_CONF` (or check out the relevant portion of the [log4j2.properties in the sources](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/config/log4j2.properties)). However, **Elastic strongly recommends using the default Log4j2 configuration**. + +Orchestrated deployments (Elastic Cloud, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK)) do not support changes in `log4j2.properties` files of the {{es}} instances. ::::{note} If you overwrite the `log4j2.properties` and do not specify appenders for any of the audit trails, audit events are forwarded to the root appender, which by default points to the `elasticsearch.log` file. :::: - +For {{es}} configuration options that control event filtering in audit logs, refer to [](./enabling-audit-logs.md#audit-logging-configuration). ## Log entry format [audit-log-entry-format] @@ -27,17 +31,4 @@ There are however a few attributes that are exceptions to the above format. The When the `request.body` attribute is present (see [Auditing search queries](auditing-search-queries.md)), it contains a string value containing the full HTTP request body, escaped as per the JSON RFC 4677. -There is a list of [audit event types](elasticsearch-audit-events.md) specifying the set of fields, as well as examples, for each entry type. - - -## Logfile output settings [audit-log-settings] - -The events and some other information about what gets logged can be controlled using settings in the `elasticsearch.yml` file. See [Audited Event Settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) and [Local Node Info Settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings). - -::::{important} -Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. -:::: - - -You can also configure how the logfile is written in the `log4j2.properties` file located in `ES_PATH_CONF` (or check out the relevant portion of the [log4j2.properties in the sources](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/config/log4j2.properties)). By default, audit information is appended to the `_audit.json` file located in the standard Elasticsearch `logs` directory (typically located at `$ES_HOME/logs`). The file is also rotated and archived daily or upon reaching the 1GB file size limit. - +Refer to [audit event types](elasticsearch-audit-events.md) for a complete list of fields, as well as examples, for each entry type. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index c0d08df8c3..b59573fb53 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -19,22 +19,39 @@ Audit logs are only available on certain [subscription levels](https://www.elast Audit logging is a powerful feature that helps you monitor and track security-related events within the {{stack}}. By enabling audit logs, you can gain visibility into authentication attempts, authorization decisions, and other system activity. -Audit logging also provides forensic evidence in the event of an attack. +Audit logging also provides forensic evidence in the event of an attack, and can be enabled independently for {{es}} and {{kib}}. Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. -This section provides a comprehensive guide to configure and use audit logging in {{es}} and {{kib}} across all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). +This section provides a comprehensive guide to [enable](./enabling-audit-logs.md) and [configure](./enabling-audit-logs.md#audit-logging-configuration) audit logging in {{es}} and {{kib}} across all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -In this section you will learn how to: +By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. -* [Enable {{es}} audit logs](./enabling-audit-logs.md#enable-audit-logging-elasticsearch). +## Optimizing audit logging for effective security monitoring [audit-logging-recommendations] -* [Enable {{kib}} audit logs](./enabling-audit-logs.md#enable-audit-logging-kibana). +When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. -* [Enable audit logs on orchestrated deployments](./enabling-audit-logs.md#enable-audit-logging-orchestrated). +To make audit logs useful and manageable to your use case, we recommend: + * Spending some time **analyzing what gets logged**. + * **Tuning the audit logging configuration** by using include/exclude filters, ignore policies, and query auditing options, to reduce unnecessary noise. Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more information. + * **Balancing verbosity with security requirements**, to ensure relevant events are captured without excessive log volume. -* [Configure audit logging](./enabling-audit-logs.md#audit-logging-configuration) to control the events that should be logged. +## Correlating audit events -* [Correlate {{kib}} and {{es}} audit events](./correlating-kibana-elasticsearch-audit-logs.md) to gain a complete view of security-related activities. +This section explains the main fields that help correlate events and understand their relationships more effectively. -By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. \ No newline at end of file +### `request.id` attribute in {{es}} audit events + +When a request generates multiple audit events across multiple nodes, you can use the `request.id` attribute to correlate the associated events. + +This identifier allows you to trace the flow of a request across the {{es}} cluster and reconstruct the full context of an operation. + +Refer to [](./elasticsearch-audit-events.md) for a complete reference of event types and attributes. + +### `trace.id` field in {{kib}} audit events + +In {{kib}}, the [trace.id](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#field-trace-id) field allows to correlate multiple events that originate from the same request. + +Additionally, this field helps correlate also events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. + +For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md) \ No newline at end of file From 838bf495b5a85b30a66dd4379461c6e8ec5b19fb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 14:14:05 +0100 Subject: [PATCH 05/27] audit logging more updates --- .../enabling-audit-logs.md | 92 +++++++------------ 1 file changed, 35 insertions(+), 57 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 09add39039..614b35fe2b 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -30,51 +30,51 @@ applies: # Enable audit logging [enable-audit-logging] +You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). + ::::{important} -Audit logs are **disabled** by default and must be explicitly enabled; they are available only on certain [subscription levels](https://www.elastic.co/subscriptions). +Audit logs are **disabled** by default and must be explicitly enabled. +:::: + +::::{tip} +Audit logs are only available on certain subscription levels. For more information, see https://www.elastic.co/subscriptions. :::: -You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -Audit logging is enabled separately for both {{es}} and {{kib}}, and enabling both is optional, depending on your use case and requirements. +Audit logging is enabled and configured independently for both {{es}} and {{kib}}, and enabling both is optional, depending on your use case and requirements. The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. The only difference lies in how the configuration is applied: - * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files. - * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. - -In short, to enable {{es}} or {{kib}} audit logs, set `xpack.security.audit.enabled` to `true` in the configuration of **all {{es}} or {{kib}} nodes**. In self-managed clusters you will have to perform a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) to apply the changes, while in orchestrated environments the orchestrator will apply the change and restart the nodes automatically. - -The following tabs show the detailed process for all deployment types: + * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files, and you will have to perform a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) when applying changes to `static` settings (such as `xpack.security.audit.enabled`). + * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator, and the restart will be applied automatically. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. ## Enabling procedure [enable-audit-logging-procedure] +In short, to enable {{es}} or {{kib}} audit logs, set `xpack.security.audit.enabled` to `true` in the configuration of **all {{es}} or {{kib}} nodes**. The following tabs show the detailed process for all deployment types: + ::::::{tab-set} :::::{tab-item} Self-managed -To enable audit logging in {{es}}: +**To enable audit logging in {{es}}**: 1. Set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. 2. Restart {{es}}. -When audit logging is enabled, audit events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). +When audit logging is enabled, security events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). -::::{note} -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html). -:::: +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](#audit-logging-configuration). -To enable audit logging in {{kib}}: +**To enable audit logging in {{kib}}**: 1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml` 2. Restart {{kib}} -::::{note} You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). -:::: + ::::: @@ -90,10 +90,6 @@ To enable audit logging in an Elastic Cloud Hosted deployment: 2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. - ::::{tip} - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - :::: - 3. From your deployment menu, go to the **Edit** page. 4. To enable auditing for Elasticsearch: @@ -105,11 +101,7 @@ To enable audit logging in an Elastic Cloud Hosted deployment: * Add the setting `xpack.security.audit.enabled: true`. 6. Select **Save changes**. - A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. - -For details on available audit settings in Elastic Cloud Hosted deployments, refer to the following documents: -* [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) -* [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) + * A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. ::::: :::::{tab-item} ECE @@ -123,9 +115,6 @@ To enable audit logging in an ECE deployment: 1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). 2. On the **Deployments** page, select your deployment. - ::::{tip} - On the deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. - :::: 3. From your deployment menu, go to the **Edit** page. @@ -139,9 +128,7 @@ To enable audit logging in an ECE deployment: * If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. 6. Select **Save**. - A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. - -For more information and other available auditing settings, refer to [{{es}} auditing settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html) and [{{kib}} auditing settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). Audit logs can be viewed within Elasticsearch and Kibana logs. + * A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. ::::: :::::{tab-item} ECK @@ -150,7 +137,7 @@ For more information and other available auditing settings, refer to [{{es}} aud In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). :::: -To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification, for example: +To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification. The following example shows this configuration together with logs and metrics delivery towards a remote cluster: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 @@ -195,23 +182,30 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref ## Advanced Configuration [audit-logging-configuration] -{{es}} audit logging and {{kib}} audit logging offer several configuration mechanisms to control what events are logged and what information is included in the audit log. For more information, refer to: +{{es}} and {{kib}} audit logging frameworks offer several configuration mechanisms to control what events are logged and what information is included in the audit log. For more information, refer to: -* [{{es}} audited event settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) +* [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) * [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings) -::::{important} -Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. -:::: +To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) described above for enabling audit logs, which applies to all deployment types. -The configuration can be applied using the same [procedure](#enable-audit-logging-procedure) used to enable audit logs, described for all deployment types. +For {{es}} `dynamic` auditing settings, you can also use the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), which is a faster and more convenient alternative that doesn't require a restart. + +::::{tip} +In {{es}}, the only auditing setting that must be added individually to all nodes and requires a restart is `xpack.security.audit.enabled`. The rest of the settings are dynamic and support being updated using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html). +:::: Configuration suggestions: -* Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the kind of events you want to include or exclude in the {{es}} auditing output. +* Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. * Refer to [logfile ignore policies](./logfile-audit-events-ignore-policies.md) for an example of how to tune the verbosity of the {{es}} audit trail. * Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. + + ::::{important} + Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. + :::: + * Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) to determine which events to exclude from the {{kib}} audit log. Extra considerations: @@ -220,22 +214,6 @@ Extra considerations: * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) -* Changing the configuration of `log4j2.properties` is not supported in orchestrated deployments, and it's not recommended for `self-managed` deployments. - -* For a complete list of event kinds and attributes, refer to: +* For a complete list of event types and attributes, refer to: * [{{es}} audit events](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) - - -## Elasticsearch audit output - - - -You can also configure how the logfile is written in the `log4j2.properties` file located in `ES_PATH_CONF` (or check out the relevant portion of the [log4j2.properties in the sources](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/config/log4j2.properties)). By default, audit information is appended to the `_audit.json` file located in the standard Elasticsearch `logs` directory (typically located at `$ES_HOME/logs`). The file is also rotated and archived daily or upon reaching the 1GB file size limit. - - - - - - - From c559ba06ac609302c1a3e7dcf286654f43574b37 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 14:41:05 +0100 Subject: [PATCH 06/27] extra changes for a draft PR --- .../monitor/logging-configuration/enabling-audit-logs.md | 3 ++- .../logfile-audit-events-ignore-policies.md | 5 ++--- .../logging-configuration/security-event-audit-logging.md | 3 ++- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 614b35fe2b..0304ed3b89 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -198,8 +198,9 @@ In {{es}}, the only auditing setting that must be added individually to all node :::: Configuration suggestions: + * Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. -* Refer to [logfile ignore policies](./logfile-audit-events-ignore-policies.md) for an example of how to tune the verbosity of the {{es}} audit trail. +* If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) to better understand how ignore policies work and when they are beneficial. * Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. ::::{important} diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md index 5c8cfbd317..857809855f 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md @@ -9,11 +9,11 @@ applies: stack: all --- -# Elasticsearch logfile audit events ignore policies [audit-log-ignore-policy] +# Elasticsearch audit events ignore policies [audit-log-ignore-policy] The comprehensive audit trail is necessary to ensure accountability. It offers tremendous value during incident response and can even be required for demonstrating compliance. -The drawback of an audited system is represented by the inevitable performance penalty incurred. In all truth, the audit trail spends *I/O ops* that are not available anymore for the user’s queries. Sometimes the verbosity of the audit trail may become a problem that the event type restrictions, [defined by `include` and `exclude`](logfile-audit-output.md#audit-log-settings), will not alleviate. +The drawback of an audited system is represented by the inevitable performance penalty incurred. In all truth, the audit trail spends *I/O ops* that are not available anymore for the user’s queries. Sometimes the verbosity of the audit trail may become a problem that the event type restrictions, [defined by `include` and `exclude`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings), will not alleviate. **Audit events ignore policies** are a finer way to tune the verbosity of the audit trail. These policies define rules that match audit events which will be *ignored* (read as: not printed). Rules match on the values of attributes of audit events and complement the `include` or `exclude` method. Imagine the corpus of audit events and the policies chopping off unwanted events. With a sole exception, all audit events are subject to the ignore policies. The exception are events of type `security_config_change`, which cannot be filtered out, unless excluded altogether. @@ -21,7 +21,6 @@ The drawback of an audited system is represented by the inevitable performance p When utilizing audit events ignore policies you are acknowledging potential accountability gaps that could render illegitimate actions undetectable. Please take time to review these policies whenever your system architecture changes. :::: - A policy is a named set of filter rules. Each filter rule applies to a single event attribute, one of the `users`, `realms`, `actions`, `roles` or `indices` attributes. The filter rule defines a list of [Lucene regexp](https://www.elastic.co/guide/en/elasticsearch/reference/current/regexp-syntax.html), **any** of which has to match the value of the audit event attribute for the rule to match. A policy matches an event if **all** the rules comprising it match the event. An audit event is ignored, therefore not printed, if it matches **any** policy. All other non-matching events are printed as usual. All policies are defined under the `xpack.security.audit.logfile.events.ignore_filters` settings namespace. For example, the following policy named *example1* matches events from the *kibana_system* or *admin_user* principals that operate over indices of the wildcard form *app-logs**: diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index b59573fb53..1577b25dca 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -32,7 +32,8 @@ By following these guidelines, you can effectively audit system behavior, enhanc When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. To make audit logs useful and manageable to your use case, we recommend: - * Spending some time **analyzing what gets logged**. + * Spending some time **analyzing what gets logged** to ensure relevant events are captured. + * **Measuring the performance impact** of audit logging on your environment, as I/O activity can be significant depending on your use case. Refer to [](./logfile-audit-events-ignore-policies.md) for more details. * **Tuning the audit logging configuration** by using include/exclude filters, ignore policies, and query auditing options, to reduce unnecessary noise. Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more information. * **Balancing verbosity with security requirements**, to ensure relevant events are captured without excessive log volume. From 94f9220e349bc4590f49d94221a92b03ad6c211e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 17:27:33 +0100 Subject: [PATCH 07/27] refining changes and removing migration comments --- .../enabling-audit-logs.md | 24 +++---------------- .../security-event-audit-logging.md | 16 ++++--------- 2 files changed, 7 insertions(+), 33 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 0304ed3b89..7f202c71db 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -13,21 +13,6 @@ applies: serverless: unavailable --- -% What needs to be done: Refine - -% GitHub issue: https://github.com/elastic/docs-projects/issues/350 - -% Scope notes: Merge the content and even consider putting everything under a global section that also covers Elasticsearch self-managed (done) - -% Use migrated content from existing pages that map to this page: - -% - [ ] ./raw-migrated-files/cloud/cloud-enterprise/ece-enable-auditing.md -% - [ ] ./raw-migrated-files/cloud-on-k8s/cloud-on-k8s/k8s_audit_logging.md -% - [ ] ./raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md - -% https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings -% https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings - # Enable audit logging [enable-audit-logging] You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). @@ -40,7 +25,6 @@ Audit logs are **disabled** by default and must be explicitly enabled. Audit logs are only available on certain subscription levels. For more information, see https://www.elastic.co/subscriptions. :::: - This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). Audit logging is enabled and configured independently for both {{es}} and {{kib}}, and enabling both is optional, depending on your use case and requirements. @@ -53,7 +37,7 @@ The only difference lies in how the configuration is applied: ## Enabling procedure [enable-audit-logging-procedure] -In short, to enable {{es}} or {{kib}} audit logs, set `xpack.security.audit.enabled` to `true` in the configuration of **all {{es}} or {{kib}} nodes**. The following tabs show the detailed process for all deployment types: +To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. The following provide detailed steps for all deployment types: ::::::{tab-set} @@ -189,12 +173,10 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) * [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings) -To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) described above for enabling audit logs, which applies to all deployment types. - -For {{es}} `dynamic` auditing settings, you can also use the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), which is a faster and more convenient alternative that doesn't require a restart. +To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. ::::{tip} -In {{es}}, the only auditing setting that must be added individually to all nodes and requires a restart is `xpack.security.audit.enabled`. The rest of the settings are dynamic and support being updated using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html). +For {{es}} auditing settings identified as `dynamic` (all except `xpack.security.audit.enabled`), you can also use the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), which is a faster and more convenient alternative that applies changes without requiring a restart. :::: Configuration suggestions: diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 1577b25dca..6957b3747e 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -7,12 +7,6 @@ applies: --- # Security event audit logging -% What needs to be done: Write from scratch -% GitHub issue: https://github.com/elastic/docs-projects/issues/350 -% Scope notes: Landing page about audit logs in Kibana and Elasticsearch, explaining how they can be enabled and configured, and also linking to the page about correlating information. We can create a doc to explain how to enable audit logging in both Elasticsearch and Kibana, and considering also ECE and orchestrated deployments. Kibana audit events list should be moved to reference content. - -% consider organizing the architecture in a better way, separating configuration pages from visualizing pages if needed... - ::::{important} Audit logs are only available on certain [subscription levels](https://www.elastic.co/subscriptions). :::: @@ -29,13 +23,11 @@ By following these guidelines, you can effectively audit system behavior, enhanc ## Optimizing audit logging for effective security monitoring [audit-logging-recommendations] -When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. +When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. + +Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more details. -To make audit logs useful and manageable to your use case, we recommend: - * Spending some time **analyzing what gets logged** to ensure relevant events are captured. - * **Measuring the performance impact** of audit logging on your environment, as I/O activity can be significant depending on your use case. Refer to [](./logfile-audit-events-ignore-policies.md) for more details. - * **Tuning the audit logging configuration** by using include/exclude filters, ignore policies, and query auditing options, to reduce unnecessary noise. Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more information. - * **Balancing verbosity with security requirements**, to ensure relevant events are captured without excessive log volume. +Balancing verbosity, performance, and security requirements is essential to capture relevant events without generating excessive log volume. ## Correlating audit events From 9cf349f9c00c663f5110d3eb1c06f2691392ae76 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 17:34:15 +0100 Subject: [PATCH 08/27] small update --- .../logging-configuration/security-event-audit-logging.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 6957b3747e..968d9b7c31 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -45,6 +45,8 @@ Refer to [](./elasticsearch-audit-events.md) for a complete reference of event t In {{kib}}, the [trace.id](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#field-trace-id) field allows to correlate multiple events that originate from the same request. -Additionally, this field helps correlate also events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. +Additionally, this field helps correlate events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. -For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md) \ No newline at end of file +For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md). + +Refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) for a complete description of {{kib}} auditing events. From 6e9e46cf92bacd3a81447836223302b716fc8079 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 6 Feb 2025 17:41:18 +0100 Subject: [PATCH 09/27] toc and titles updated --- .../monitor/logging-configuration/auditing-search-queries.md | 2 +- .../monitor/logging-configuration/logfile-audit-output.md | 2 +- deploy-manage/toc.yml | 5 ++--- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index 16907b747d..12937a601d 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -8,7 +8,7 @@ applies: stack: all --- -# Auditing search queries [auditing-search-queries] +# Auditing Elasticsearch search queries [auditing-search-queries] There is no [audit event type](elasticsearch-audit-events.md) specifically dedicated to search queries. Search queries are analyzed and then processed; the processing triggers authorization actions that are audited. However, the original raw query, as submitted by the client, is not accessible downstream when authorization auditing occurs. diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index 5474ff18c0..9ccd17dd83 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -1,5 +1,5 @@ --- -navigation_title: Elasticsearch audit output +navigation_title: Elasticsearch logfile output mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/audit-log-output.html applies: diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 52f186c644..2d410fb081 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -780,12 +780,11 @@ toc: - file: monitor/logging-configuration/security-event-audit-logging.md children: - file: monitor/logging-configuration/enabling-audit-logs.md - children: - - file: monitor/logging-configuration/elasticsearch-audit-events.md - - file: monitor/logging-configuration/logfile-audit-output.md - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - file: monitor/logging-configuration/auditing-search-queries.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md + - file: monitor/logging-configuration/elasticsearch-audit-events.md + - file: monitor/logging-configuration/logfile-audit-output.md - file: cloud-organization.md children: - file: cloud-organization/billing.md From dc389b4b8a3a4907558dc316076271a92d95a9b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 08:52:01 +0100 Subject: [PATCH 10/27] enable audit log simplified a bit --- .../enabling-audit-logs.md | 64 ++++++++++++------- 1 file changed, 40 insertions(+), 24 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 7f202c71db..85a35017fa 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -15,7 +15,7 @@ applies: # Enable audit logging [enable-audit-logging] -You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). +You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). Audit logging can be enabled independently for {{es}} and {{kib}}. ::::{important} Audit logs are **disabled** by default and must be explicitly enabled. @@ -27,10 +27,10 @@ Audit logs are only available on certain subscription levels. For more informati This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -Audit logging is enabled and configured independently for both {{es}} and {{kib}}, and enabling both is optional, depending on your use case and requirements. - +% consider removing this sentence as maybe it doesn't add any benefit The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. +% consider removing this sentence as maybe it doesn't add any benefit The only difference lies in how the configuration is applied: * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files, and you will have to perform a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) when applying changes to `static` settings (such as `xpack.security.audit.enabled`). * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator, and the restart will be applied automatically. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. @@ -168,35 +168,51 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref {{es}} and {{kib}} audit logging frameworks offer several configuration mechanisms to control what events are logged and what information is included in the audit log. For more information, refer to: -* [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) -* [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) -* [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) -* [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings) +### Elasticsearch auditing configuration -To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. +**{{es}} configuration** options include: -::::{tip} -For {{es}} auditing settings identified as `dynamic` (all except `xpack.security.audit.enabled`), you can also use the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), which is a faster and more convenient alternative that applies changes without requiring a restart. -:::: + * [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) + * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) + * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) -Configuration suggestions: + ::::{tip} + In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**, which means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) as a faster and more convenient method that applies changes without requiring a restart. + :::: -* Use [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. -* If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) to better understand how ignore policies work and when they are beneficial. -* Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. +For more information about event details and format, refer to: + * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). + * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#log-entry-format-audit-log-entry-format) - ::::{important} - Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. - :::: +* Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: + * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) + +### Kibana auditing configuration + +**{{kib}} configuration** options include: + + * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) -* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) to determine which events to exclude from the {{kib}} audit log. + ::::{tip} + To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. + :::: -Extra considerations: +For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to: + * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) * Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: - * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) -* For a complete list of event types and attributes, refer to: - * [{{es}} audit events](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). - * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) +### Configuration suggestions + +* Consider starting with [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. + +* If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) for a better understanding how ignore policies work and when they are beneficial. + +* Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. + + ::::{important} + Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. + :::: + +* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) if you want to filter out certain events from the {{kib}} audit log. \ No newline at end of file From 3bc8233ecbb13c17a28bbc9dd0e4abf4d7ae3d9a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 08:53:06 +0100 Subject: [PATCH 11/27] added serverless unavailble tags --- .../logging-configuration/security-event-audit-logging.md | 1 + 1 file changed, 1 insertion(+) diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 968d9b7c31..84eb3b4eff 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -4,6 +4,7 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- # Security event audit logging From c4347ef6a3ba3aa47ecca9390b97b15e99ec2b13 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 08:55:21 +0100 Subject: [PATCH 12/27] link fixed --- .../monitor/logging-configuration/enabling-audit-logs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 85a35017fa..c5a9740940 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -182,7 +182,7 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref For more information about event details and format, refer to: * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). - * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#log-entry-format-audit-log-entry-format) + * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) * Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) From 2c2517d601cc7271174cc0ac31fc24a957a34b17 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 23:25:01 +0100 Subject: [PATCH 13/27] renaming again with tool to fix links --- .../monitor/logging-configuration/enabling-audit-logs.md | 2 +- deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md | 2 +- .../users-roles/cluster-or-deployment-auth/internal-users.md | 2 +- .../ingest-data-with-nodejs-on-elasticsearch-service.md | 2 +- .../ingest-data-with-python-on-elasticsearch-service.md | 2 +- .../cloud/cloud-enterprise/ece-getting-started-node-js.md | 2 +- .../cloud/cloud-enterprise/ece-getting-started-python.md | 2 +- .../cloud/cloud-heroku/ech-enable-logging-and-monitoring.md | 2 +- .../cloud/cloud/ec-enable-logging-and-monitoring.md | 2 +- raw-migrated-files/cloud/cloud/ec-getting-started-node-js.md | 2 +- raw-migrated-files/cloud/cloud/ec-getting-started-python.md | 2 +- .../elasticsearch/elasticsearch-reference/secure-cluster.md | 2 +- troubleshoot/elasticsearch/hotspotting.md | 2 +- 13 files changed, 13 insertions(+), 13 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index c5a9740940..594de2e503 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -96,7 +96,7 @@ In orchestrated deployments, audit logs must be shipped to a monitoring deployme To enable audit logging in an ECE deployment: -1. [Log into the Cloud UI](../../../deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui.md). +1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. On the **Deployments** page, select your deployment. diff --git a/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md b/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md index 5aa4f0c630..a8238b8f8d 100644 --- a/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md +++ b/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md @@ -180,7 +180,7 @@ When shipping logs to a monitoring deployment there are more logging features av #### For {{es}}: [ece-extra-logging-features-elasticsearch] -* [Audit logging](../logging-configuration/enabling-elasticsearch-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment * [Slow query and index logging](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html) - helps find and debug slow queries and indexing * Verbose logging - helps debug stack issues by increasing component logs diff --git a/deploy-manage/users-roles/cluster-or-deployment-auth/internal-users.md b/deploy-manage/users-roles/cluster-or-deployment-auth/internal-users.md index 41c9fe1bb1..42f2a3785a 100644 --- a/deploy-manage/users-roles/cluster-or-deployment-auth/internal-users.md +++ b/deploy-manage/users-roles/cluster-or-deployment-auth/internal-users.md @@ -14,5 +14,5 @@ The {{stack-security-features}} use eight *internal* users (`_system`, `_xpack`, These users are only used by requests that originate from within the cluster. For this reason, they cannot be used to authenticate against the API and there is no password to manage or reset. -From time-to-time you may find a reference to one of these users inside your logs, including [audit logs](../../monitor/logging-configuration/enabling-elasticsearch-audit-logs.md). +From time-to-time you may find a reference to one of these users inside your logs, including [audit logs](../../monitor/logging-configuration/enabling-audit-logs.md). diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md index 378831465e..49366bdb4a 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md @@ -290,7 +290,7 @@ const client = new Client({ }) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). ### Best practices [ec_best_practices] diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md index ecb9630fb8..08ac3b8bbc 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md @@ -351,7 +351,7 @@ es = Elasticsearch( ) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/examples.html). diff --git a/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-node-js.md b/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-node-js.md index 3cacc5f79b..a17d9805f9 100644 --- a/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-node-js.md +++ b/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-node-js.md @@ -280,7 +280,7 @@ const client = new Client({ }) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elastic Cloud Enterprise, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elastic Cloud Enterprise, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). ### Best practices [ece_best_practices] diff --git a/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-python.md b/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-python.md index 716d5d889a..06bc882290 100644 --- a/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-python.md +++ b/raw-migrated-files/cloud/cloud-enterprise/ece-getting-started-python.md @@ -340,7 +340,7 @@ es = Elasticsearch( ) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elastic Cloud Enterprise, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elastic Cloud Enterprise, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/examples.html). diff --git a/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md b/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md index 6dc86b9606..cb0bad7d5d 100644 --- a/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md +++ b/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md @@ -172,7 +172,7 @@ When shipping logs to a monitoring deployment there are more logging features av #### For {{es}}: [ech-extra-logging-features-elasticsearch] -* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment * [Slow query and index logging](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html) - helps find and debug slow queries and indexing * Verbose logging - helps debug stack issues by increasing component logs diff --git a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md index 2c2a378284..a55111e7bc 100644 --- a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md +++ b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md @@ -172,7 +172,7 @@ When shipping logs to a monitoring deployment there are more logging features av #### For {{es}}: [ec-extra-logging-features-elasticsearch] -* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment * [Slow query and index logging](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html) - helps find and debug slow queries and indexing * Verbose logging - helps debug stack issues by increasing component logs diff --git a/raw-migrated-files/cloud/cloud/ec-getting-started-node-js.md b/raw-migrated-files/cloud/cloud/ec-getting-started-node-js.md index 3d74b083cd..bae5215fdc 100644 --- a/raw-migrated-files/cloud/cloud/ec-getting-started-node-js.md +++ b/raw-migrated-files/cloud/cloud/ec-getting-started-node-js.md @@ -272,7 +272,7 @@ const client = new Client({ }) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). ### Best practices [ec_best_practices] diff --git a/raw-migrated-files/cloud/cloud/ec-getting-started-python.md b/raw-migrated-files/cloud/cloud/ec-getting-started-python.md index 0a3d33cb52..35d42892e1 100644 --- a/raw-migrated-files/cloud/cloud/ec-getting-started-python.md +++ b/raw-migrated-files/cloud/cloud/ec-getting-started-python.md @@ -333,7 +333,7 @@ es = Elasticsearch( ) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/examples.html). diff --git a/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-cluster.md b/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-cluster.md index be6cdb49ca..fd450af95e 100644 --- a/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-cluster.md +++ b/raw-migrated-files/elasticsearch/elasticsearch-reference/secure-cluster.md @@ -42,5 +42,5 @@ See [Configure security for the {{stack}}](../../../deploy-manage/security/secur Keeping a system secure takes vigilance. By using {{stack-security-features}} to maintain an audit trail, you can easily see who is accessing your cluster and what they’re doing. You can configure the audit level, which accounts for the type of events that are logged. These events include failed authentication attempts, user access denied, node connection denied, and more. By analyzing access patterns and failed attempts to access your cluster, you can gain insights into attempted attacks and data breaches. Keeping an auditable log of the activity in your cluster can also help diagnose operational issues. -See [Enable audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md). +See [Enable audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md). diff --git a/troubleshoot/elasticsearch/hotspotting.md b/troubleshoot/elasticsearch/hotspotting.md index 0dfe02d9e3..dc84608774 100644 --- a/troubleshoot/elasticsearch/hotspotting.md +++ b/troubleshoot/elasticsearch/hotspotting.md @@ -175,5 +175,5 @@ Its response contains a `description` that reports this query: indices[winlogbeat-*,logs-window*], sequence by winlog.computer_name with maxspan=1m\n\n[authentication where host.os.type == "windows" and event.action:"logged-in" and\n event.outcome == "success" and process.name == "svchost.exe" ] by winlog.event_data.TargetLogonId ``` -This lets you know which indices to check (`winlogbeat-*,logs-window*`), as well as the [EQL search](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-search-api.html) request body. Most likely this is [SIEM related](https://www.elastic.co/guide/en/security/current/es-overview.html). You can combine this with [audit logging](../../deploy-manage/monitor/logging-configuration/enabling-elasticsearch-audit-logs.md) as needed to trace the request source. +This lets you know which indices to check (`winlogbeat-*,logs-window*`), as well as the [EQL search](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-search-api.html) request body. Most likely this is [SIEM related](https://www.elastic.co/guide/en/security/current/es-overview.html). You can combine this with [audit logging](../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) as needed to trace the request source. From b4d38b7180f532dd98b7db641ca4b5aa5788b526 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 23:28:56 +0100 Subject: [PATCH 14/27] links fixed due to kibana audit logs file deleted --- deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md | 2 +- .../cloud/cloud-heroku/ech-enable-logging-and-monitoring.md | 2 +- .../cloud/cloud/ec-enable-logging-and-monitoring.md | 2 +- raw-migrated-files/kibana/kibana/introduction.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md b/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md index a8238b8f8d..c55d2fb997 100644 --- a/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md +++ b/deploy-manage/monitor/stack-monitoring/ece-stack-monitoring.md @@ -189,7 +189,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic #### For Kibana: [ece-extra-logging-features-kibana] -* [Audit logging](../logging-configuration/enabling-kibana-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../deploy/cloud-enterprise/edit-stack-settings.md) to enable this feature. diff --git a/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md b/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md index cb0bad7d5d..360735a76f 100644 --- a/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md +++ b/raw-migrated-files/cloud/cloud-heroku/ech-enable-logging-and-monitoring.md @@ -181,7 +181,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic #### For Kibana: [ech-extra-logging-features-kibana] -* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../../deploy-manage/deploy/elastic-cloud/edit-stack-settings.md) to enable this feature. diff --git a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md index a55111e7bc..5160df6f1a 100644 --- a/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md +++ b/raw-migrated-files/cloud/cloud/ec-enable-logging-and-monitoring.md @@ -181,7 +181,7 @@ After you’ve enabled log delivery on your deployment, you can [add the Elastic #### For Kibana: [ec-extra-logging-features-kibana] -* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md) - logs security-related events on your deployment +* [Audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) - logs security-related events on your deployment After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](../../../deploy-manage/deploy/elastic-cloud/edit-stack-settings.md) to enable this feature. diff --git a/raw-migrated-files/kibana/kibana/introduction.md b/raw-migrated-files/kibana/kibana/introduction.md index 5a2cf318c4..683c3e3052 100644 --- a/raw-migrated-files/kibana/kibana/introduction.md +++ b/raw-migrated-files/kibana/kibana/introduction.md @@ -129,7 +129,7 @@ The {{kib}} role management interface allows you to describe these various acces ### Audit access [_audit_access] -Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, see [{{kib}} audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-kibana-audit-logs.md). +Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, see [{{kib}} audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md). ## Find apps and objects [kibana-navigation-search] From d4491dd51dd41f468ca35f7af9f52a1bee7f57a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Fri, 7 Feb 2025 23:58:43 +0100 Subject: [PATCH 15/27] subscription feature note updated --- .../monitor/logging-configuration/enabling-audit-logs.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 594de2e503..d5a055233a 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -18,11 +18,11 @@ applies: You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). Audit logging can be enabled independently for {{es}} and {{kib}}. ::::{important} -Audit logs are **disabled** by default and must be explicitly enabled. +Audit logs are only available on certain [subscription levels](https://www.elastic.co/subscriptions). :::: -::::{tip} -Audit logs are only available on certain subscription levels. For more information, see https://www.elastic.co/subscriptions. +::::{note} +Audit logs are **disabled** by default and must be explicitly enabled. :::: This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). From 7e86fe4fedd8cdf8fd2fd64f5d039dba378e6afc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 12 Feb 2025 21:17:15 +0100 Subject: [PATCH 16/27] fixed link in get-started/introduction.md file --- get-started/introduction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/get-started/introduction.md b/get-started/introduction.md index 4099bded7b..3b1196a77f 100644 --- a/get-started/introduction.md +++ b/get-started/introduction.md @@ -129,7 +129,7 @@ The {{kib}} role management interface allows you to describe these various acces ### Audit access [_audit_access] -Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, see [{{kib}} audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md). +Once you have your users and roles configured, you might want to maintain a record of who did what, when. The {{kib}} audit log will record this information for you, which can then be correlated with {{es}} audit logs to gain more insights into your users’ behavior. For more information, see [{{kib}} audit logging](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md). ## Find apps and objects [kibana-navigation-search] From 1b4dd3f8c381a9244dad10ffc480b37cb87287e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 12 Feb 2025 21:21:54 +0100 Subject: [PATCH 17/27] Update manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md --- .../ingest-data-with-nodejs-on-elasticsearch-service.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md index 1383aa2700..9ebcd4a44b 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md @@ -290,7 +290,7 @@ const client = new Client({ }) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). ### Best practices [ec_best_practices] From 542f5a5c537d59ec782ccd9e5f643d0d7270341d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Wed, 12 Feb 2025 21:22:59 +0100 Subject: [PATCH 18/27] Update manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md --- .../ingest-data-with-python-on-elasticsearch-service.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md index 73cb034c3e..e760e1de07 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md @@ -351,7 +351,7 @@ es = Elasticsearch( ) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on Elasticsearch Service, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/examples.html). From e852bd412e71104f7e158ef929ea59da789afdae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 13 Feb 2025 11:28:27 +0100 Subject: [PATCH 19/27] changes applied per psanz suggestions --- .../auditing-search-queries.md | 2 +- .../configuring-audit-logs.md | 61 +++++++++++++ ...elating-kibana-elasticsearch-audit-logs.md | 34 ++++++++ .../enabling-audit-logs.md | 85 ++++--------------- .../security-event-audit-logging.md | 36 ++------ deploy-manage/toc.yml | 6 +- 6 files changed, 123 insertions(+), 101 deletions(-) create mode 100644 deploy-manage/monitor/logging-configuration/configuring-audit-logs.md diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index 12937a601d..39666598ab 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -8,7 +8,7 @@ applies: stack: all --- -# Auditing Elasticsearch search queries [auditing-search-queries] +# Audit Elasticsearch search queries [auditing-search-queries] There is no [audit event type](elasticsearch-audit-events.md) specifically dedicated to search queries. Search queries are analyzed and then processed; the processing triggers authorization actions that are audited. However, the original raw query, as submitted by the client, is not accessible downstream when authorization auditing occurs. diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md new file mode 100644 index 0000000000..59fa409148 --- /dev/null +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -0,0 +1,61 @@ +--- +applies: + hosted: all + ece: all + eck: all + stack: all + serverless: unavailable +--- + +# Configure audit logging [audit-logging-configuration] + +When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. + +### Elasticsearch auditing configuration + +**{{es}} configuration** options include: + + * [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) + * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) + * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) + + ::::{tip} + In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**, which means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) as a faster and more convenient method that applies changes without requiring a restart. + :::: + +Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: + * [Elasticsearch audit settings for Elastic Cloud Hosted deployments](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) + +For a complete description of event details and format, refer to: + * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). + * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) + +### Kibana auditing configuration + +**{{kib}} configuration** options include: + + * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) + + ::::{tip} + To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. + :::: + +Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: + * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) + +For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to: + * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) + +### General recommendations + +* Consider starting with [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. + +* If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) for a better understanding how ignore policies work and when they are beneficial. + +* Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. + + ::::{important} + Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. + :::: + +* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) if you want to filter out certain events from the {{kib}} audit log. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index f2b72b4b95..0d23265b88 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -10,6 +10,40 @@ applies: --- # Correlating Kibana and Elasticsearch audit events [xpack-security-ecs-audit-correlation] +(TBD) + +When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. + +Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more details. + +Balancing verbosity, performance, and security requirements is essential to capture relevant events without generating excessive log volume. + +## Optimizing audit logging for effective security monitoring [audit-logging-recommendations] + + +## Correlating audit events + +This section explains the main fields that help correlate events and understand their relationships more effectively. + +### `request.id` attribute in {{es}} audit events + +When a request generates multiple audit events across multiple nodes, you can use the `request.id` attribute to correlate the associated events. + +This identifier allows you to trace the flow of a request across the {{es}} cluster and reconstruct the full context of an operation. + +Refer to [](./elasticsearch-audit-events.md) for a complete reference of event types and attributes. + +### `trace.id` field in {{kib}} audit events + +In {{kib}}, the [trace.id](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#field-trace-id) field allows to correlate multiple events that originate from the same request. + +Additionally, this field helps correlate events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. + +For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md). + +Refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) for a complete description of {{kib}} auditing events. + + Audit events can be correlated in two ways: diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index d5a055233a..0d49625b41 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -15,29 +15,31 @@ applies: # Enable audit logging [enable-audit-logging] -You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). Audit logging can be enabled independently for {{es}} and {{kib}}. - ::::{important} Audit logs are only available on certain [subscription levels](https://www.elastic.co/subscriptions). :::: +You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). Audit logging can be enabled independently for {{es}} and {{kib}}. + +Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. + ::::{note} Audit logs are **disabled** by default and must be explicitly enabled. :::: This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -% consider removing this sentence as maybe it doesn't add any benefit -The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. - -% consider removing this sentence as maybe it doesn't add any benefit -The only difference lies in how the configuration is applied: - * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files, and you will have to perform a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) when applying changes to `static` settings (such as `xpack.security.audit.enabled`). - * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator, and the restart will be applied automatically. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. - ## Enabling procedure [enable-audit-logging-procedure] -To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. The following provide detailed steps for all deployment types: +To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. The following provide detailed steps for all supported deployment types: + +% content discarded (for review) +% The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. +% +% consider removing this sentence as maybe it doesn't add any benefit +% The only difference lies in how the configuration is applied: +% * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files, and you will have to perform %a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) when applying changes to `static` settings (such as `xpack.security.audit.enabled`). +% * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator, and the restart will be applied automatically. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. ::::::{tab-set} @@ -45,12 +47,12 @@ To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` **To enable audit logging in {{es}}**: -1. Set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. -2. Restart {{es}}. +1. In all nodes, set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. +2. Restart the cluster by following the [rolling restart](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) procedure. When audit logging is enabled, security events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](#audit-logging-configuration). +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](./configuring-audit-logs.md). **To enable audit logging in {{kib}}**: @@ -59,7 +61,6 @@ You can configure additional options to control what events are logged and what You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). - ::::: :::::{tab-item} Elastic Cloud Hosted @@ -68,7 +69,7 @@ You can optionally configure audit logs location, file/rolling file appenders an In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). :::: -To enable audit logging in an Elastic Cloud Hosted deployment: +To enable audit logging in an {{ech}} deployment: 1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). @@ -164,55 +165,3 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref :::::: -## Advanced Configuration [audit-logging-configuration] - -{{es}} and {{kib}} audit logging frameworks offer several configuration mechanisms to control what events are logged and what information is included in the audit log. For more information, refer to: - -### Elasticsearch auditing configuration - -**{{es}} configuration** options include: - - * [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) - * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) - * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) - - ::::{tip} - In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**, which means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) as a faster and more convenient method that applies changes without requiring a restart. - :::: - -For more information about event details and format, refer to: - * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). - * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) - -* Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: - * [Elasticsearch audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) - -### Kibana auditing configuration - -**{{kib}} configuration** options include: - - * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) - - ::::{tip} - To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. - :::: - -For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to: - * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) - -* Elastic Cloud Hosted deployments provide its own subset of supported settings for auditing configuration: - * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) - -### Configuration suggestions - -* Consider starting with [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. - -* If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) for a better understanding how ignore policies work and when they are beneficial. - -* Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. - - ::::{important} - Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. - :::: - -* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) if you want to filter out certain events from the {{kib}} audit log. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 84eb3b4eff..945c374952 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -16,38 +16,14 @@ Audit logging is a powerful feature that helps you monitor and track security-re Audit logging also provides forensic evidence in the event of an attack, and can be enabled independently for {{es}} and {{kib}}. -Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. +In this section, you'll learn how to: -This section provides a comprehensive guide to [enable](./enabling-audit-logs.md) and [configure](./enabling-audit-logs.md#audit-logging-configuration) audit logging in {{es}} and {{kib}} across all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). +1. [](./enabling-audit-logs.md): Activate {{es}} or {{kib}} audit logs for all supported deployment types, including self-managed clusters, {{ech}}, {{ece}} (ECE), and {{eck}} (ECK). -By following these guidelines, you can effectively audit system behavior, enhance security monitoring, and meet compliance requirements. +2. [](./configuring-audit-logs.md): Filter and control what security events get logged in the audit log output. -## Optimizing audit logging for effective security monitoring [audit-logging-recommendations] +3. Optionally, [audit {{es}} search queries](./auditing-search-queries.md): Audit and log search request bodies. -When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. +4. [Correlate audit events](./correlating-kibana-elasticsearch-audit-logs.md): Explore audit logs and understand how events from the same request are correlated. -Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more details. - -Balancing verbosity, performance, and security requirements is essential to capture relevant events without generating excessive log volume. - -## Correlating audit events - -This section explains the main fields that help correlate events and understand their relationships more effectively. - -### `request.id` attribute in {{es}} audit events - -When a request generates multiple audit events across multiple nodes, you can use the `request.id` attribute to correlate the associated events. - -This identifier allows you to trace the flow of a request across the {{es}} cluster and reconstruct the full context of an operation. - -Refer to [](./elasticsearch-audit-events.md) for a complete reference of event types and attributes. - -### `trace.id` field in {{kib}} audit events - -In {{kib}}, the [trace.id](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#field-trace-id) field allows to correlate multiple events that originate from the same request. - -Additionally, this field helps correlate events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. - -For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md). - -Refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) for a complete description of {{kib}} auditing events. +By following these guidelines, you can effectively audit system activity, enhance security monitoring, and meet compliance requirements. diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 377271f23f..6c0606ee26 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -780,8 +780,10 @@ toc: - file: monitor/logging-configuration/security-event-audit-logging.md children: - file: monitor/logging-configuration/enabling-audit-logs.md - - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - - file: monitor/logging-configuration/auditing-search-queries.md + - file: monitor/logging-configuration/configuring-audit-logs.md + children: + - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md + - file: monitor/logging-configuration/auditing-search-queries.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md - file: monitor/logging-configuration/elasticsearch-audit-events.md - file: monitor/logging-configuration/logfile-audit-output.md From 433fdbcc544378484c1be4eef5b7829caa30d61c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 13 Feb 2025 11:36:05 +0100 Subject: [PATCH 20/27] fixing links --- .../monitor/logging-configuration/configuring-audit-logs.md | 2 +- .../correlating-kibana-elasticsearch-audit-logs.md | 2 +- .../monitor/logging-configuration/logfile-audit-output.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index 59fa409148..a3a8d98135 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -37,7 +37,7 @@ For a complete description of event details and format, refer to: * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) ::::{tip} - To configure {{kib}} settings, follow the same [procedure](#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. + To configure {{kib}} settings, follow the same [procedure](./enabling-audit-logs.md#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. :::: Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index 0d23265b88..b5d13d5ee8 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -14,7 +14,7 @@ applies: When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. -Refer to [](./enabling-audit-logs.md#audit-logging-configuration) for more details. +Refer to [](./configuring-audit-logs.md) for more details. Balancing verbosity, performance, and security requirements is essential to capture relevant events without generating excessive log volume. diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index 9ccd17dd83..a003a89a6e 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -21,7 +21,7 @@ Orchestrated deployments (Elastic Cloud, Elastic Cloud Enterprise (ECE), and Ela If you overwrite the `log4j2.properties` and do not specify appenders for any of the audit trails, audit events are forwarded to the root appender, which by default points to the `elasticsearch.log` file. :::: -For {{es}} configuration options that control event filtering in audit logs, refer to [](./enabling-audit-logs.md#audit-logging-configuration). +For {{es}} configuration options that control event filtering in audit logs, refer to [](./configuring-audit-logs.md). ## Log entry format [audit-log-entry-format] From a107e954f8afdd7ccc0645d4d54f55b2618c05a5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 13 Feb 2025 13:16:08 +0100 Subject: [PATCH 21/27] correlating events updated --- .../configuring-audit-logs.md | 2 +- ...elating-kibana-elasticsearch-audit-logs.md | 32 ++++++------------- 2 files changed, 10 insertions(+), 24 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index a3a8d98135..1955d2ac42 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -20,7 +20,7 @@ When auditing security events, a single client request might generate multiple a * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) ::::{tip} - In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**, which means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html) as a faster and more convenient method that applies changes without requiring a restart. + In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**. This means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), allowing changes to take effect immediately without requiring a restart. This approach is faster and more convenient than modifying `elasticsearch.yml`. :::: Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index b5d13d5ee8..036b7d8f7f 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -9,52 +9,38 @@ applies: stack: all --- -# Correlating Kibana and Elasticsearch audit events [xpack-security-ecs-audit-correlation] -(TBD) +# Correlating audit events [xpack-security-ecs-audit-correlation] -When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. +When audit logs are enabled, a single request to {{kib}} or {{es}} generates multiple audit events in the logs. -Refer to [](./configuring-audit-logs.md) for more details. +Audit events from {{kib}} can also be correlated with backend calls that produce {{es}} audit events, allowing for a more comprehensive view of user actions. -Balancing verbosity, performance, and security requirements is essential to capture relevant events without generating excessive log volume. +This section explains the key fields that help correlate these events, with examples to illustrate their relationships. -## Optimizing audit logging for effective security monitoring [audit-logging-recommendations] +## `request.id` attribute in {{es}} audit events - -## Correlating audit events - -This section explains the main fields that help correlate events and understand their relationships more effectively. - -### `request.id` attribute in {{es}} audit events - -When a request generates multiple audit events across multiple nodes, you can use the `request.id` attribute to correlate the associated events. +When an {{es}} request generates multiple audit events across multiple nodes, you can use the `request.id` attribute to correlate the associated events. This identifier allows you to trace the flow of a request across the {{es}} cluster and reconstruct the full context of an operation. Refer to [](./elasticsearch-audit-events.md) for a complete reference of event types and attributes. -### `trace.id` field in {{kib}} audit events +## `trace.id` field in {{kib}} audit events In {{kib}}, the [trace.id](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#field-trace-id) field allows to correlate multiple events that originate from the same request. Additionally, this field helps correlate events from one request with the backend calls that create {{es}} audit events. When {{kib}} sends requests to {{es}}, the `trace.id` value is propagated and stored in the `opaque_id` attribute of {{es}} audit logs, allowing cross-component correlation. -For an example of correlating {{es}} and {{kib}} audit logs, refer to [](./correlating-kibana-elasticsearch-audit-logs.md). - Refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) for a complete description of {{kib}} auditing events. +## Examples - -Audit events can be correlated in two ways: - -1. Multiple {{kib}} audit events that resulted from the same request can be correlated together. -2. If [{{es}} audit logging](enabling-audit-logs.md) is enabled, {{kib}} audit events from one request can be correlated with backend calls that create {{es}} audit events. +This section shows practical examples of correlating audit logs. ::::{note} The examples below are simplified, many fields have been omitted and values have been shortened for clarity. :::: - ### Example 1: correlating multiple {{kib}} audit events [_example_1_correlating_multiple_kib_audit_events] When "thom" creates a new alerting rule, five audit events are written: From 55db7cbadf40eccfecf49a503b5b8165cce22cbc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 13 Feb 2025 13:30:03 +0100 Subject: [PATCH 22/27] audit request body updated --- .../monitor/logging-configuration/auditing-search-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index 39666598ab..287649c68b 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -20,7 +20,7 @@ To make certain audit events include the request body, configure the following s xpack.security.audit.logfile.events.emit_request_body: true ``` -In self-managed deployments, add the to the `elasticsearch.yml` file, while in orchestrated environments, apply the setting through the appropriate [configuration mechanism](/deploy-manage/deploy.md). +You can apply this setting through [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), as described in [](./configuring-audit-logs.md). Alternatively, you can modify `elasticsearch.yml` in all nodes and restart for the changes to take effect. ::::{important} No filtering is performed when auditing, so **sensitive data might be audited in plain text when audit events include the request body**. Also, the request body can contain malicious content that can break a parser consuming the audit logs. From f801787dfacf213df026a6d787237ffa942a85bd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Thu, 13 Feb 2025 14:15:59 +0100 Subject: [PATCH 23/27] minor changes, ready for review --- .../configuring-audit-logs.md | 14 +++++++------- .../elasticsearch-audit-events.md | 2 -- deploy-manage/toc.yml | 2 +- 3 files changed, 8 insertions(+), 10 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index 1955d2ac42..a6b0f7c555 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -9,15 +9,15 @@ applies: # Configure audit logging [audit-logging-configuration] -When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and *I/O operations*. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. +When auditing security events, a single client request might generate multiple audit events across multiple cluster nodes, potentially leading to a high volume of log data and I/O operations. To maintain clarity and ensure logs remain actionable, {{es}} and {{kib}} provide configuration mechanisms to control what events are logged and which can be ignored. ### Elasticsearch auditing configuration -**{{es}} configuration** options include: +{{es}} configuration options include: - * [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) - * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings) - * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies) + * [{{es}} audited events settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings): Use include and exclude filters to control the types of events that get logged. + * [{{es}} node information settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#node-audit-settings): Control whether to add or hide node information such as hostname or IP address in the audited events. + * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies): Use ignore policies for fine-grained control over which audit events are printed to the log file. ::::{tip} In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**. This means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), allowing changes to take effect immediately without requiring a restart. This approach is faster and more convenient than modifying `elasticsearch.yml`. @@ -32,9 +32,9 @@ For a complete description of event details and format, refer to: ### Kibana auditing configuration -**{{kib}} configuration** options include: +{{kib}} configuration options include: - * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) + * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters): List of filters that determine which events should be excluded from the audit log. ::::{tip} To configure {{kib}} settings, follow the same [procedure](./enabling-audit-logs.md#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. diff --git a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md index a56f0cae9e..072ee198b3 100644 --- a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md +++ b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md @@ -12,8 +12,6 @@ applies: # Elasticsearch audit events [elasticsearch-audit-events] -When you are [auditing security events](/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md), a single client request might generate multiple audit events across multiple cluster nodes. The common `request.id` attribute can be used to correlate the associated events. - This document provides a reference for all audit event [types](#audit-event-types) and their associated [attributes](#audit-event-attributes) in {{es}}. Types and attributes can be used to control what gets logged through [audit event settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) and [ignore policies](/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md). ::::{note} diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index 6c0606ee26..c43ffea159 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -784,9 +784,9 @@ toc: children: - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - file: monitor/logging-configuration/auditing-search-queries.md + - file: monitor/logging-configuration/logfile-audit-output.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md - file: monitor/logging-configuration/elasticsearch-audit-events.md - - file: monitor/logging-configuration/logfile-audit-output.md - file: cloud-organization.md children: - file: cloud-organization/billing.md From df94d2ccca0f7db596a2ce85efa47d6b4585ce77 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Mon, 17 Feb 2025 08:29:56 +0100 Subject: [PATCH 24/27] Apply suggestions from code review 29 suggestions from Shaina Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --- .../auditing-search-queries.md | 13 +++++- .../configuring-audit-logs.md | 18 +++----- .../enabling-audit-logs.md | 43 +++++++++---------- .../logfile-audit-output.md | 2 +- .../security-event-audit-logging.md | 8 ++-- ...ta-with-nodejs-on-elasticsearch-service.md | 2 +- ...ta-with-python-on-elasticsearch-service.md | 2 +- 7 files changed, 45 insertions(+), 43 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index 287649c68b..dafcf2b6e9 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -23,12 +23,21 @@ xpack.security.audit.logfile.events.emit_request_body: true You can apply this setting through [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), as described in [](./configuring-audit-logs.md). Alternatively, you can modify `elasticsearch.yml` in all nodes and restart for the changes to take effect. ::::{important} -No filtering is performed when auditing, so **sensitive data might be audited in plain text when audit events include the request body**. Also, the request body can contain malicious content that can break a parser consuming the audit logs. +No filtering is performed when auditing, so sensitive data might be audited in plain text when audit events include the request body. Also, the request body can contain malicious content that can break a parser consuming the audit logs. :::: The request body is printed as an escaped JSON string value (RFC 4627) to the `request.body` event attribute. -Not all events contain the `request.body` attribute, even when the above setting is toggled. The ones that do are: `authentication_success`, `authentication_failed`, `realm_authentication_failed`, `tampered_request`, `run_as_denied`, and `anonymous_access_denied`. The `request.body` attribute is printed on the coordinating node only (the node that handles the REST request). Most of these event types are [not included by default](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include). +Not all events contain the `request.body` attribute, even when the above setting is toggled. The ones that do are: + +* `authentication_success` +* `authentication_failed` +* `realm_authentication_failed` +* `tampered_request` +* `run_as_denied` +* `anonymous_access_denied` + +The `request.body` attribute is printed on the coordinating node only (the node that handles the REST request). Most of these event types are [not included by default](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include). A good practical piece of advice is to add `authentication_success` to the event types that are audited (add it to the list in the `xpack.security.audit.logfile.events.include`), as this event type is not audited by default. diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index a6b0f7c555..9ccfc8ba91 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -20,28 +20,22 @@ When auditing security events, a single client request might generate multiple a * [{{es}} ignore policies settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#audit-event-ignore-policies): Use ignore policies for fine-grained control over which audit events are printed to the log file. ::::{tip} - In {{es}}, all auditing settings except `xpack.security.audit.enabled` are **dynamic**. This means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), allowing changes to take effect immediately without requiring a restart. This approach is faster and more convenient than modifying `elasticsearch.yml`. + In {{es}}, all auditing settings except `xpack.security.audit.enabled` are dynamic. This means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), allowing changes to take effect immediately without requiring a restart. This approach is faster and more convenient than modifying `elasticsearch.yml`. :::: -Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: - * [Elasticsearch audit settings for Elastic Cloud Hosted deployments](https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html#ec_audit_settings) -For a complete description of event details and format, refer to: - * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md). - * [{{es}} logentry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) +For a complete description of event details and format, refer to the following resources: + * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md) + * [{{es}} log entry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) ### Kibana auditing configuration -{{kib}} configuration options include: - - * [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters): List of filters that determine which events should be excluded from the audit log. +To control the logs that are outputted by Kibana, you can use [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters). These are a list of filters that determine which events should be excluded from the audit log. ::::{tip} To configure {{kib}} settings, follow the same [procedure](./enabling-audit-logs.md#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. :::: -Note that {{ech}} deployments provide its own subset of supported settings for auditing configuration: - * [Kibana audit settings on Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-manage-kibana-settings.html#ec_logging_and_audit_settings) For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to: * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) @@ -55,7 +49,7 @@ For a complete description of auditing event details, such as `category`, `type` * Refer to [auditing search queries](./auditing-search-queries.md) for details on logging request bodies in the {{es}} audit logs. ::::{important} - Be advised that **sensitive data may be audited in plain text** when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. + Sensitive data may be audited in plain text when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. :::: * Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) if you want to filter out certain events from the {{kib}} audit log. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 0d49625b41..912acf95cf 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -24,14 +24,18 @@ You can log security-related events such as authentication failures and refused Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. ::::{note} -Audit logs are **disabled** by default and must be explicitly enabled. +Audit logs are disabled by default and must be explicitly enabled. :::: This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). -## Enabling procedure [enable-audit-logging-procedure] +::::{important} +In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). +:::: + +## Enable audit logging [enable-audit-logging-procedure] -To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. The following provide detailed steps for all supported deployment types: +To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. For detailed instructions, select your deployment type: % content discarded (for review) % The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. @@ -56,8 +60,8 @@ You can configure additional options to control what events are logged and what **To enable audit logging in {{kib}}**: -1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml` -2. Restart {{kib}} +1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml`. +2. Restart {{kib}}. You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). @@ -65,20 +69,17 @@ You can optionally configure audit logs location, file/rolling file appenders an :::::{tab-item} Elastic Cloud Hosted -::::{important} -In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). -:::: To enable audit logging in an {{ech}} deployment: -1. Log in to the [Elasticsearch Service Console](https://cloud.elastic.co?page=docs&placement=docs-body). +1. Log in to the [{{ecloud}} Console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. Find your deployment on the home page in the Elasticsearch Service card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. +2. Find your deployment on the home page in the **Hosted deployments** card and select **Manage** to access it directly. Or, select **Hosted deployments** to go to the deployments page to view all of your deployments. 3. From your deployment menu, go to the **Edit** page. 4. To enable auditing for Elasticsearch: - * In the **Elasticsearch** section select **Manage user settings and extensions**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for each node instead. + * In the **Elasticsearch** section, select **Manage user settings and extensions**. For deployments with existing user settings, you may have to expand the **Edit elasticsearch.yml** caret for each node instead. * Add the setting `xpack.security.audit.enabled: true`. 5. To enable auditing for Kibana: @@ -86,18 +87,16 @@ To enable audit logging in an {{ech}} deployment: * Add the setting `xpack.security.audit.enabled: true`. 6. Select **Save changes**. - * A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. + +A plan change will run on your deployment. When it finishes, audit logs will be delivered to your monitoring deployment. ::::: :::::{tab-item} ECE -::::{important} -In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). -:::: To enable audit logging in an ECE deployment: -1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). +1. [Log in to the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md). 2. On the **Deployments** page, select your deployment. @@ -113,16 +112,16 @@ To enable audit logging in an ECE deployment: * If your Elastic Stack version is below 7.6.0, add the setting `logging.quiet: false`. 6. Select **Save**. - * A plan change will run on your deployment. When it finishes, **audit logs will be delivered to your monitoring deployment**. + +A plan change will run on your deployment. When it finishes, audit logs will be delivered to your monitoring deployment. ::::: :::::{tab-item} ECK -::::{important} -In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). -:::: -To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification. The following example shows this configuration together with logs and metrics delivery towards a remote cluster: +To enable audit logging in an ECK-managed cluster, add `xpack.security.audit.enabled: true` to the `config` section of each {{es}} `nodeSet` and to the `config` section of the {{kib}} object's specification. + +The following example shows this configuration, along with together with logs and metrics delivery towards a remote cluster: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 @@ -160,7 +159,7 @@ spec: xpack.security.audit.enabled: true ``` -When enabled, audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section. If monitoring is not enabled audit logs will only be visible at container level. +When enabled, audit logs are collected and shipped to the monitoring cluster referenced in the `monitoring.logs` section. If monitoring is not enabled, audit logs will only be visible at container level. ::::: :::::: diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index a003a89a6e..b8e73169a1 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -15,7 +15,7 @@ The `logfile` audit output is the only output for auditing. By default, it write In self-managed clusters, you can configure how the `logfile` is written in the `log4j2.properties` file located in `ES_PATH_CONF` (or check out the relevant portion of the [log4j2.properties in the sources](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/config/log4j2.properties)). However, **Elastic strongly recommends using the default Log4j2 configuration**. -Orchestrated deployments (Elastic Cloud, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK)) do not support changes in `log4j2.properties` files of the {{es}} instances. +Orchestrated deployments (ECH, ECE, and ECK) do not support changes in `log4j2.properties` files of the {{es}} instances. ::::{note} If you overwrite the `log4j2.properties` and do not specify appenders for any of the audit trails, audit events are forwarded to the root appender, which by default points to the `elasticsearch.log` file. diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 945c374952..5c41bb75d3 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -18,12 +18,12 @@ Audit logging also provides forensic evidence in the event of an attack, and can In this section, you'll learn how to: -1. [](./enabling-audit-logs.md): Activate {{es}} or {{kib}} audit logs for all supported deployment types, including self-managed clusters, {{ech}}, {{ece}} (ECE), and {{eck}} (ECK). +* [](./enabling-audit-logs.md): Activate {{es}} or {{kib}} audit logs for all supported deployment types. -2. [](./configuring-audit-logs.md): Filter and control what security events get logged in the audit log output. +* [](./configuring-audit-logs.md): Filter and control what security events get logged in the audit log output. -3. Optionally, [audit {{es}} search queries](./auditing-search-queries.md): Audit and log search request bodies. +* [Audit {{es}} search queries](./auditing-search-queries.md): Audit and log search request bodies. -4. [Correlate audit events](./correlating-kibana-elasticsearch-audit-logs.md): Explore audit logs and understand how events from the same request are correlated. +* [Correlate audit events](./correlating-kibana-elasticsearch-audit-logs.md): Explore audit logs and understand how events from the same request are correlated. By following these guidelines, you can effectively audit system activity, enhance security monitoring, and meet compliance requirements. diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md index 9ebcd4a44b..81f0643e4e 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-nodejs-on-elasticsearch-service.md @@ -290,7 +290,7 @@ const client = new Client({ }) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). ### Best practices [ec_best_practices] diff --git a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md index e760e1de07..0349583259 100644 --- a/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md +++ b/manage-data/ingest/ingesting-data-from-applications/ingest-data-with-python-on-elasticsearch-service.md @@ -351,7 +351,7 @@ es = Elasticsearch( ) ``` -Check [Create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). +Check [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) to learn more about API Keys and [Security privileges](../../../deploy-manage/users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to understand which privileges are needed. If you are not sure what the right combination of privileges for your custom application is, you can enable [audit logging](../../../deploy-manage/monitor/logging-configuration/enabling-audit-logs.md) on {{es}} to find out what privileges are being used. To learn more about how logging works on {{ech}} or {{ece}}, check [Monitoring Elastic Cloud deployment logs and metrics](https://www.elastic.co/blog/monitoring-elastic-cloud-deployment-logs-and-metrics). For more information on refreshing an index, searching, updating, and deleting, check the [elasticsearch-py examples](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/examples.html). From 79e676067e079b8d640e5af5ad6725bd7ae6a010 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Mon, 17 Feb 2025 08:31:52 +0100 Subject: [PATCH 25/27] Update deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com> --- .../correlating-kibana-elasticsearch-audit-logs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index 036b7d8f7f..fc804a4dc9 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -38,7 +38,7 @@ Refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/x This section shows practical examples of correlating audit logs. ::::{note} -The examples below are simplified, many fields have been omitted and values have been shortened for clarity. +The examples below are simplified. Many fields have been omitted and values have been shortened for clarity. :::: ### Example 1: correlating multiple {{kib}} audit events [_example_1_correlating_multiple_kib_audit_events] From 24d1154a2a49fbd257c6db1a6e1f01dce908a7fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Mon, 17 Feb 2025 08:56:38 +0100 Subject: [PATCH 26/27] changes post Shaina's review --- .../configuring-audit-logs.md | 16 +++++++--------- .../enabling-audit-logs.md | 18 ++++-------------- 2 files changed, 11 insertions(+), 23 deletions(-) diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index 9ccfc8ba91..1e3e4a8583 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -23,7 +23,6 @@ When auditing security events, a single client request might generate multiple a In {{es}}, all auditing settings except `xpack.security.audit.enabled` are dynamic. This means you can configure them using the [cluster update settings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html), allowing changes to take effect immediately without requiring a restart. This approach is faster and more convenient than modifying `elasticsearch.yml`. :::: - For a complete description of event details and format, refer to the following resources: * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md) * [{{es}} log entry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) @@ -32,17 +31,18 @@ For a complete description of event details and format, refer to the following r To control the logs that are outputted by Kibana, you can use [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters). These are a list of filters that determine which events should be excluded from the audit log. - ::::{tip} - To configure {{kib}} settings, follow the same [procedure](./enabling-audit-logs.md#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. - :::: +In self-managed systems, you can optionally configure audit logs location, and file/rolling file using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). -For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to: - * [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging) +::::{tip} +To configure {{kib}} settings, follow the same [procedure](./enabling-audit-logs.md#enable-audit-logging-procedure) as when enabling {{kib}} audit logs, but apply the relevant settings instead. +:::: + +For a complete description of auditing event details, such as `category`, `type`, or `action`, refer to [{{kib}} audit events](https://www.elastic.co/guide/en/kibana/current/xpack-security-audit-logging.html#xpack-security-ecs-audit-logging). ### General recommendations -* Consider starting with [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) or the corresponding `exclude` setting to specify the type of events you want to include or exclude in the {{es}} auditing output. +* Consider starting with {{es}} [`xpack.security.audit.logfile.events.include`](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#xpack-sa-lf-events-include) and [{{kib}} ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) settings to specify the type of events you want to include or exclude in the auditing output. * If you need a more granular control, refer to [{{es}} audit events ignore policies](./logfile-audit-events-ignore-policies.md) for a better understanding how ignore policies work and when they are beneficial. @@ -51,5 +51,3 @@ For a complete description of auditing event details, such as `category`, `type` ::::{important} Sensitive data may be audited in plain text when including the request body in audit events, even though all the security APIs, such as those that change the user’s password, have the credentials filtered out when audited. :::: - -* Use {{kib}} [ignore filters](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-ignore-filters) if you want to filter out certain events from the {{kib}} audit log. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index 912acf95cf..c2ab49bcd4 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -33,18 +33,14 @@ This section describes how to enable and configure audit logging in both {{es}} In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). :::: +When audit logging is enabled, security events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). + +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](./configuring-audit-logs.md). + ## Enable audit logging [enable-audit-logging-procedure] To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. For detailed instructions, select your deployment type: -% content discarded (for review) -% The process of enabling and configuring audit logging is consistent across all supported deployment types, whether self-managed, Elastic Cloud, Elastic Cloud Enterprise (ECE), or Elastic Cloud on Kubernetes (ECK). The same settings apply regardless of the deployment type, ensuring a unified approach to audit logging configuration. -% -% consider removing this sentence as maybe it doesn't add any benefit -% The only difference lies in how the configuration is applied: -% * In self-managed clusters, settings are added directly to the `elasticsearch.yml` and `kibana.yml` configuration files, and you will have to perform %a [rolling restart of the cluster](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) when applying changes to `static` settings (such as `xpack.security.audit.enabled`). -% * In orchestrated deployments (Elastic Cloud Hosted, ECE or ECK), the configuration is applied using the appropriate mechanisms provided by the orchestrator, and the restart will be applied automatically. Refer to [](/deploy-manage/deploy.md) for more information about the different orchestrators and deployment configuration mechanisms. - ::::::{tab-set} :::::{tab-item} Self-managed @@ -54,17 +50,11 @@ To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` 1. In all nodes, set `xpack.security.audit.enabled` to `true` in `elasticsearch.yml`. 2. Restart the cluster by following the [rolling restart](/deploy-manage/maintenance/start-stop-services/full-cluster-restart-rolling-restart-procedures.md) procedure. -When audit logging is enabled, security events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). - -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](./configuring-audit-logs.md). - **To enable audit logging in {{kib}}**: 1. Set `xpack.security.audit.enabled` to `true` in `kibana.yml`. 2. Restart {{kib}}. -You can optionally configure audit logs location, file/rolling file appenders and ignore filters using [{{kib}} audit logging settings](https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#audit-logging-settings). - ::::: :::::{tab-item} Elastic Cloud Hosted From 664a9feaf606193e83a5f6f9424bbea889df61db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edu=20Gonz=C3=A1lez=20de=20la=20Herr=C3=A1n?= <25320357+eedugon@users.noreply.github.com> Date: Tue, 18 Feb 2025 12:13:43 +0100 Subject: [PATCH 27/27] elasticsearch audit details removed and asciidocalypse temp links added --- .../auditing-search-queries.md | 3 +- .../configuring-audit-logs.md | 2 +- ...elating-kibana-elasticsearch-audit-logs.md | 3 +- .../elasticsearch-audit-events.md | 899 ------------------ .../enabling-audit-logs.md | 18 +- .../logfile-audit-events-ignore-policies.md | 3 +- .../logfile-audit-output.md | 5 +- .../security-event-audit-logging.md | 7 + deploy-manage/toc.yml | 5 +- 9 files changed, 28 insertions(+), 917 deletions(-) delete mode 100644 deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md diff --git a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md index dafcf2b6e9..6c609be5b3 100644 --- a/deploy-manage/monitor/logging-configuration/auditing-search-queries.md +++ b/deploy-manage/monitor/logging-configuration/auditing-search-queries.md @@ -6,11 +6,12 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- # Audit Elasticsearch search queries [auditing-search-queries] -There is no [audit event type](elasticsearch-audit-events.md) specifically dedicated to search queries. Search queries are analyzed and then processed; the processing triggers authorization actions that are audited. However, the original raw query, as submitted by the client, is not accessible downstream when authorization auditing occurs. +There is no [audit event type]() (asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events) specifically dedicated to search queries. Search queries are analyzed and then processed; the processing triggers authorization actions that are audited. However, the original raw query, as submitted by the client, is not accessible downstream when authorization auditing occurs. Search queries are contained inside HTTP request bodies, however, and some audit events that are generated by the REST layer, on the coordinating node, can be toggled to output the request body to the audit log. Therefore, one must audit request bodies in order to audit search queries. diff --git a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md index 1e3e4a8583..03f84ee07b 100644 --- a/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/configuring-audit-logs.md @@ -24,7 +24,7 @@ When auditing security events, a single client request might generate multiple a :::: For a complete description of event details and format, refer to the following resources: - * [{{es}} audit events details and schema](/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md) + * [{{es}} audit events details and schema]() asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events * [{{es}} log entry output format](/deploy-manage/monitor/logging-configuration/logfile-audit-output.md#audit-log-entry-format) ### Kibana auditing configuration diff --git a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md index fc804a4dc9..25c0e2ca6d 100644 --- a/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md @@ -7,6 +7,7 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- # Correlating audit events [xpack-security-ecs-audit-correlation] @@ -23,7 +24,7 @@ When an {{es}} request generates multiple audit events across multiple nodes, yo This identifier allows you to trace the flow of a request across the {{es}} cluster and reconstruct the full context of an operation. -Refer to [](./elasticsearch-audit-events.md) for a complete reference of event types and attributes. +Refer to [linkTBD]() asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events for a complete reference of event types and attributes. ## `trace.id` field in {{kib}} audit events diff --git a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md b/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md deleted file mode 100644 index 072ee198b3..0000000000 --- a/deploy-manage/monitor/logging-configuration/elasticsearch-audit-events.md +++ /dev/null @@ -1,899 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/elasticsearch/reference/current/audit-event-types.html -applies: - hosted: all - ece: all - eck: all - stack: all ---- - -% Pending to move this page to REFERENCE - -# Elasticsearch audit events [elasticsearch-audit-events] - -This document provides a reference for all audit event [types](#audit-event-types) and their associated [attributes](#audit-event-attributes) in {{es}}. Types and attributes can be used to control what gets logged through [audit event settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/auditing-settings.html#event-audit-settings) and [ignore policies](/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md). - -::::{note} -Certain audit events require the `security_config_change` event type to log the related event action. The event descriptions in this document indicate whether this requirement is applicable. -:::: - -## Audit event types [audit-event-types] - -$$$event-access-denied$$$ - -`access_denied` -: Logged when an authenticated user attempts to execute an action they do not have the necessary [privilege](../../users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to perform. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:30:06,949+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"transport", "event.action": - "access_denied", "authentication.type":"REALM", "user.name":"user1", - "user.realm":"default_native", "user.roles":["test_role"], "origin.type": - "rest", "origin.address":"[::1]:52434", "request.id":"yKOgWn2CRQCKYgZRz3phJw", - "action":"indices:admin/auto_create", "request.name":"CreateIndexRequest", - "indices":[""]} - ``` - - :::: - - -$$$event-access-granted$$$ - -`access_granted` -: Logged when an authenticated user attempts to execute an action they have the necessary privilege to perform. These events will be logged only for non-system users. - - If you want to include `access_granted` events for all users (including internal users such as `_xpack`), add [`system_access_granted`](#event-system-granted) to the list of event types in addition to `access_granted`. The `system_access_granted` privilege is not included by default to avoid cluttering the logs. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:30:06,947+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"transport", "event.action": - "access_granted", "authentication.type":"REALM", "user.name":"user1", "user - realm":"default_native", "user.roles":["test_role"], "origin.type":"rest", - "origin.address":"[::1]:52434", "request.id":"yKOgWn2CRQCKYgZRz3phJw", - "action":"indices:data/write/bulk", "request.name":"BulkRequest"} - ``` - - :::: - - -$$$event-anonymous-access-denied$$$ - -`anonymous_access_denied` -: Logged when a request is denied due to missing authentication credentials. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T21:56:43,608+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"rest", "event.action": - "anonymous_access_denied", "origin.type":"rest", "origin.address": - "[::1]:50543", "url.path":"/twitter/_async_search", "url.query":"pretty", - "request.method":"POST", "request.id":"TqA9OisyQ8WTl1ivJUV1AA"} - ``` - - :::: - - -$$$event-authentication-failed$$$ - -`authentication_failed` -: Logged when the authentication credentials cannot be matched to a known user. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:10:15,510+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"rest", "event.action": - "authentication_failed", "user.name":"elastic", "origin.type":"rest", - "origin.address":"[::1]:51504", "url.path":"/_security/user/user1", - "url.query":"pretty", "request.method":"POST", - "request.id":"POv8p_qeTl2tb5xoFl0HIg"} - ``` - - :::: - - -$$$event-authentication-success$$$ - -`authentication_success` -: Logged when a user successfully authenticates. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:03:35,018+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"rest", "event.action": - "authentication_success", "authentication.type":"REALM", "user.name": - "elastic", "user.realm":"reserved", "origin.type":"rest", "origin.address": - "[::1]:51014", "realm":"reserved", "url.path":"/twitter/_search", - "url.query":"pretty", "request.method":"POST", - "request.id":"nHV3UMOoSiu-TaSPWCfxGg"} - ``` - - :::: - - -$$$event-change-disable-user$$$ - -`change_disable_user` -: Logged when the [enable user API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-enable-user.html) is invoked to disable a native or a built-in user. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T23:17:28,308+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"change_disable_user", "request.id":"qvLIgw_eTvyK3cgV-GaLVg", - "change":{"disable":{"user":{"name":"user1"}}}} - ``` - - :::: - - -$$$event-change-enable-user$$$ - -`change_enable_user` -: Logged when the [enable user API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-enable-user.html) is invoked to enable a native or a built-in user. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T23:17:34,843+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"change_enable_user", "request.id":"BO3QU3qeTb-Ei0G0rUOalQ", - "change":{"enable":{"user":{"name":"user1"}}}} - ``` - - :::: - - -$$$event-change-password$$$ - -`change_password` -: Logged when the [change password API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-change-password.html) is invoked to change the password of a native or built-in user. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2019-12-30T22:19:41,345+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"change_password", "request.id":"bz5a1Cc3RrebDMitMGGNCw", - "change":{"password":{"user":{"name":"user1"}}}} - ``` - - :::: - - -$$$event-create-service-token$$$ - -`create_service_token` -: Logged when the [create service account token API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-service-token.html) is invoked to create a new index-based token for a service account. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2021-04-30T23:17:42,952+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"create_service_token", "request.id":"az9a1Db5QrebDMacQ8yGKc", - "create":{"service_token":{"namespace":"elastic","service":"fleet-server","name":"token1"}}}` - ``` - - :::: - - -$$$event-connection-denied$$$ - -`connection_denied` -: Logged when an incoming TCP connection does not pass the [IP filter](../../security/ip-traffic-filtering.md) for a specific profile. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T21:47:31,526+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"ip_filter", "event.action": - "connection_denied", "origin.type":"rest", "origin.address":"10.10.0.20:52314", - "transport.profile":".http", "rule":"deny 10.10.0.0/16"} - ``` - - :::: - - -$$$event-connection-granted$$$ - -`connection_granted` -: Logged when an incoming TCP connection passes the [IP filter](../../security/ip-traffic-filtering.md) for a specific profile. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T21:47:31,526+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"ip_filter", "event.action": - "connection_granted", "origin.type":"rest", "origin.address":"[::1]:52314", - "transport.profile":".http", "rule":"allow ::1,127.0.0.1"} - ``` - - :::: - - -$$$event-create-apikey$$$ - -`create_apikey` -: Logged when the [create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) or the [grant API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-grant-api-key.html) APIs are invoked to create a new API key. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:33:52,521+0200", "node.id": - "9clhpgjJRR-iKzOw20xBNQ", "event.type":"security_config_change", "event.action": - "create_apikey", "request.id":"9FteCmovTzWHVI-9Gpa_vQ", "create":{"apikey": - {"name":"test-api-key-1","expiration":"10d","role_descriptors":[{"cluster": - ["monitor","manage_ilm"],"indices":[{"names":["index-a*"],"privileges": - ["read","maintenance"]},{"names":["in*","alias*"],"privileges":["read"], - "field_security":{"grant":["field1*","@timestamp"],"except":["field11"]}}], - "applications":[],"run_as":[]},{"cluster":["all"],"indices":[{"names": - ["index-b*"],"privileges":["all"]}],"applications":[],"run_as":[]}], - "metadata":{"application":"my-application","environment":{"level": 1, - "tags":["dev","staging"]}}}}} - ``` - - :::: - - -$$$event-change-apikey$$$ - -`change_apikey` -: Logged when the [update API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-update-api-key.html) API is invoked to update the attributes of an existing API key. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:33:52,521+0200", "node.id": - "9clhpgjJRR-iKzOw20xBNQ", "event.type":"security_config_change", "event.action": - "change_apikey", "request.id":"9FteCmovTzWHVI-9Gpa_vQ", "change":{"apikey": - {"id":"zcwN3YEBBmnjw-K-hW5_","role_descriptors":[{"cluster": - ["monitor","manage_ilm"],"indices":[{"names":["index-a*"],"privileges": - ["read","maintenance"]},{"names":["in*","alias*"],"privileges":["read"], - "field_security":{"grant":["field1*","@timestamp"],"except":["field11"]}}], - "applications":[],"run_as":[]},{"cluster":["all"],"indices":[{"names": - ["index-b*"],"privileges":["all"]}],"applications":[],"run_as":[]}], - "metadata":{"application":"my-application","environment":{"level": 1, - "tags":["dev","staging"]}},"expiration":"10d"}}} - ``` - - :::: - - -$$$event-change-apikeys$$$ - -`change_apikeys` -: Logged when the [bulk update API keys](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-bulk-update-api-keys.html) API is invoked to update the attributes of multiple existing API keys. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit","timestamp":"2020-12-31T00:33:52,521+0200","node.id": - "9clhpgjJRR-iKzOw20xBNQ","event.type":"security_config_change", - "event.action":"change_apikeys","request.id":"9FteCmovTzWHVI-9Gpa_vQ", - "change":{"apikeys": - {"ids":["zcwN3YEBBmnjw-K-hW5_","j7c0WYIBqecB5CbVR6Oq"],"role_descriptors": - [{"cluster":["monitor","manage_ilm"],"indices":[{"names":["index-a*"],"privileges": - ["read","maintenance"]},{"names":["in*","alias*"],"privileges":["read"], - "field_security":{"grant":["field1*","@timestamp"],"except":["field11"]}}], - "applications":[],"run_as":[]},{"cluster":["all"],"indices":[{"names": - ["index-b*"],"privileges":["all"]}],"applications":[],"run_as":[]}], - "metadata":{"application":"my-application","environment":{"level":1, - "tags":["dev","staging"]}},"expiration":"10d"}}} - ``` - - :::: - - -$$$event-delete-privileges$$$ - -`delete_privileges` -: Logged when the [delete application privileges API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-delete-privilege.html) is invoked to remove one or more application privileges. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:39:30,246+0200", "node.id": - "9clhpgjJRR-iKzOw20xBNQ", "event.type":"security_config_change", "event. - action":"delete_privileges", "request.id":"7wRWVxxqTzCKEspeSP7J8g", - "delete":{"privileges":{"application":"myapp","privileges":["read"]}}} - ``` - - :::: - - -$$$event-delete-role$$$ - -`delete_role` -: Logged when the [delete role API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-delete-role.html) is invoked to delete a role. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:08:11,678+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event.action": - "delete_role", "request.id":"155IKq3zQdWq-12dgKZRnw", - "delete":{"role":{"name":"my_admin_role"}}} - ``` - - :::: - - -$$$event-delete-role-mapping$$$ - -`delete_role_mapping` -: Logged when the [delete role mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-delete-role-mapping.html) is invoked to delete a role mapping. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:12:09,349+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"delete_role_mapping", "request.id":"Stim-DuoSTCWom0S_xhf8g", - "delete":{"role_mapping":{"name":"mapping1"}}} - ``` - - :::: - - -$$$event-delete-service-token$$$ - -`delete_service_token` -: Logged when the [delete service account token API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-delete-service-token.html) is invoked to delete an index-based token for a service account. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2021-04-30T23:17:42,952+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"delete_service_token", "request.id":"az9a1Db5QrebDMacQ8yGKc", - "delete":{"service_token":{"namespace":"elastic","service":"fleet-server","name":"token1"}}} - ``` - - :::: - - -$$$event-delete-user$$$ - -`delete_user` -: Logged when the [delete user API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-delete-user.html) is invoked to delete a specific native user. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:19:41,345+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", - "event.action":"delete_user", "request.id":"au5a1Cc3RrebDMitMGGNCw", - "delete":{"user":{"name":"jacknich"}}} - ``` - - :::: - - -$$$event-invalidate-apikeys$$$ - -`invalidate_apikeys` -: Logged when the [invalidate API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-invalidate-api-key.html) is invoked to invalidate one or more API keys. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:36:30,247+0200", "node.id": - "9clhpgjJRR-iKzOw20xBNQ", "event.type":"security_config_change", "event. - action":"invalidate_apikeys", "request.id":"7lyIQU9QTFqSrTxD0CqnTQ", - "invalidate":{"apikeys":{"owned_by_authenticated_user":false, - "user":{"name":"myuser","realm":"native1"}}}} - ``` - - :::: - - -$$$event-put-privileges$$$ - -`put_privileges` -: Logged when the [create or update privileges API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-privileges.html) is invoked to add or update one or more application privileges. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:39:07,779+0200", "node.id": - "9clhpgjJRR-iKzOw20xBNQ", "event.type":"security_config_change", - "event.action":"put_privileges", "request.id":"1X2VVtNgRYO7FmE0nR_BGA", - "put":{"privileges":[{"application":"myapp","name":"read","actions": - ["data:read/*","action:login"],"metadata":{"description":"Read access to myapp"}}]}} - ``` - - :::: - - -$$$event-put-role$$$ - -`put_role` -: Logged when the [create or update role API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role.html) is invoked to create or update a role. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:27:01,978+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", - "event.action":"put_role", "request.id":"tDYQhv5CRMWM4Sc5Zkk2cQ", - "put":{"role":{"name":"test_role","role_descriptor":{"cluster":["all"], - "indices":[{"names":["apm*"],"privileges":["all"],"field_security": - {"grant":["granted"]},"query":"{\"term\": {\"service.name\": \"bar\"}}"}, - {"names":["apm-all*"],"privileges":["all"],"query":"{\"term\": - {\"service.name\": \"bar2\"}}"}],"applications":[],"run_as":[]}}}} - ``` - - :::: - - -$$$event-put-role-mapping$$$ - -`put_role_mapping` -: Logged when the [create or update role mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role-mapping.html) is invoked to create or update a role mapping. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-31T00:11:13,932+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", "event. - action":"put_role_mapping", "request.id":"kg4h1l_kTDegnLC-0A-XxA", - "put":{"role_mapping":{"name":"mapping1","roles":["user"],"rules": - {"field":{"username":"*"}},"enabled":true,"metadata":{"version":1}}}} - ``` - - :::: - - -$$$event-put-user$$$ - -`put_user` -: Logged when the [create or update user API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-user.html) is invoked to create or update a native user. Note that user updates can also change the user’s password. - - You must include the `security_config_change` event type to audit the related event action. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:10:09,749+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"security_config_change", - "event.action":"put_user", "request.id":"VIiSvhp4Riim_tpkQCVSQA", - "put":{"user":{"name":"user1","enabled":false,"roles":["admin","other_role1"], - "full_name":"Jack Sparrow","email":"jack@blackpearl.com", - "has_password":true,"metadata":{"cunning":10}}}} - ``` - - :::: - - -$$$event-realm-auth-failed$$$ - -`realm_authentication_failed` -: Logged for every realm that fails to present a valid authentication token. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:10:15,510+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"rest", "event.action": - "realm_authentication_failed", "user.name":"elastic", "origin.type":"rest", - "origin.address":"[::1]:51504", "realm":"myTestRealm1", "url.path": - "/_security/user/user1", "url.query":"pretty", "request.method":"POST", - "request.id":"POv8p_qeTl2tb5xoFl0HIg"} - ``` - - :::: - - -$$$event-runas-denied$$$ - -`run_as_denied` -: Logged when an authenticated user attempts to [run as](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) another user that they do not have the necessary [privileges](../../users-roles/cluster-or-deployment-auth/elasticsearch-privileges.md) to do so. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:49:34,859+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"transport", "event.action": - "run_as_denied", "user.name":"user1", "user.run_as.name":"user1", - "user.realm":"default_native", "user.run_as.realm":"default_native", - "user.roles":["test_role"], "origin.type":"rest", "origin.address": - "[::1]:52662", "request.id":"RcaSt872RG-R_WJBEGfYXA", - "action":"indices:data/read/search", "request.name":"SearchRequest", "indices":["alias1"]} - ``` - - :::: - - -$$$event-runas-granted$$$ - -`run_as_granted` -: Logged when an authenticated user attempts to [run as](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) another user that they have the necessary privileges to do so. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2020-12-30T22:44:42,068+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type":"transport", "event.action": - "run_as_granted", "user.name":"elastic", "user.run_as.name":"user1", - "user.realm":"reserved", "user.run_as.realm":"default_native", - "user.roles":["superuser"], "origin.type":"rest", "origin.address": - "[::1]:52623", "request.id":"dGqPTdEQSX2TAPS3cvc1qA", "action": - "indices:data/read/search", "request.name":"SearchRequest", "indices":["alias1"]} - ``` - - :::: - - -$$$event-system-granted$$$ - -`system_access_granted` -: Logs [`access_granted`](#event-access-granted) events only for [internal users](../../users-roles/cluster-or-deployment-auth/internal-users.md), such as `_xpack`. If you include this setting in addition to `access_granted`, then `access_granted` events are logged for *all* users. - - ::::{note} - This event type is disabled by default to avoid cluttering the logs. - :::: - - -$$$event-tampered-request$$$ - -`tampered_request` -: Logged when the {{security-features}} detect that the request has been tampered with. Typically relates to `search/scroll` requests when the scroll ID is believed to have been tampered with. - - ::::{dropdown} Example - ```js - {"type":"audit", "timestamp":"2019-11-27T22:00:00,947+0200", "node.id": - "0RMNyghkQYCc_gVd1G6tZQ", "event.type": "rest", "event.action": - "tampered_request", "origin.address":"[::1]:50543", "url.path": - "/twitter/_async_search", "url.query":"pretty", "request.method":"POST", - "request.id":"TqA9OisyQ8WTl1ivJUV1AA"} - ``` - - :::: - - - -## Audit event attributes [audit-event-attributes] - -The audit events are formatted as JSON documents, and each event is printed on a separate line in the audit log. The entries themselves do not contain an end-of-line delimiter. For more details, see [Log entry format](logfile-audit-output.md#audit-log-entry-format). - -The following list shows attributes that are common to all audit event types: - -`@timestamp` -: The time, in ISO9601 format, when the event occurred. - -`node.name` -: The name of the node. This can be changed in the `elasticsearch.yml` config file. - -`node.id` -: The node id. This is automatically generated and is persistent across full cluster restarts. - -`host.ip` -: The bound IP address of the node, with which the node can be communicated with. - -`host.name` -: The unresolved node’s hostname. - -`event.type` -: The internal processing layer that generated the event: `rest`, `transport`, `ip_filter` or `security_config_change`. This is different from `origin.type` because a request originating from the REST API is translated to a number of transport messages, generating audit events with `origin.type: rest` and `event.type: transport`. - -`event.action` -: The type of event that occurred: `anonymous_access_denied`, `authentication_failed`, `authentication_success`, `realm_authentication_failed`, `access_denied`, `access_granted`, `connection_denied`, `connection_granted`, `tampered_request`, `run_as_denied`, or `run_as_granted`. - - In addition, if `event.type` equals [`security_config_change`](#security-config-change), the `event.action` attribute takes one of the following values: `put_user`, `change_password`, `put_role`, `put_role_mapping`, `change_enable_user`, `change_disable_user`, `put_privileges`, `create_apikey`, `delete_user`, `delete_role`, `delete_role_mapping`, `invalidate_apikeys`, `delete_privileges`, `change_apikey`, or `change_apikeys`. - - -`request.id` -: A synthetic identifier that can be used to correlate the events associated with a particular REST request. - -In addition, all the events of types `rest`, `transport` and `ip_filter` (but not `security_config_change`) have the following extra attributes, which show more details about the requesting client: - -`origin.address` -: The source IP address of the request associated with this event. This could be the address of the remote client, the address of another cluster node, or the local node’s bound address, if the request originated locally. Unless the remote client connects directly to the cluster, the *client address* will actually be the address of the first OSI layer 3 proxy in front of the cluster. - -`origin.type` -: The origin type of the request associated with this event: `rest` (request originated from a REST API request), `transport` (request was received on the transport channel), or `local_node` (the local node issued the request). - -`opaque_id` -: The value of the `X-Opaque-Id` HTTP header (if present) of the request associated with this event. See more: [`X-Opaque-Id` HTTP header - API conventions](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#x-opaque-id) - -`trace_id` -: The identifier extracted from the `traceparent` HTTP header (if present) of the request associated with this event. It allows to surface audit logs into the Trace Logs feature of Elastic APM. - -`x_forwarded_for` -: The verbatim value of the `X-Forwarded-For` HTTP request header (if present) of the request associated with the audit event. This header is commonly added by proxies when they forward requests and the value is the address of the proxied client. When a request crosses multiple proxies the header is a comma delimited list with the last value being the address of the second to last proxy server (the address of the last proxy server is designated by the `origin.address` field). - -## Audit event attributes of the `rest` event type [_audit_event_attributes_of_the_rest_event_type] - -The events with `event.type` equal to `rest` have one of the following `event.action` attribute values: `authentication_success`, `anonymous_access_denied`, `authentication_failed`, `realm_authentication_failed`, `tampered_request` or `run_as_denied`. These events also have the following extra attributes (in addition to the common ones): - -`url.path` -: The path part of the URL (between the port and the query string) of the REST request associated with this event. This is URL encoded. - -`url.query` -: The query part of the URL (after "?", if present) of the REST request associated with this event. This is URL encoded. - -`request.method` -: The HTTP method of the REST request associated with this event. It is one of GET, POST, PUT, DELETE, OPTIONS, HEAD, PATCH, TRACE and CONNECT. - -`request.body` -: The full content of the REST request associated with this event, if enabled. This contains the HTTP request body. The body is escaped as a string value according to the JSON RFC 4627. - - -## Audit event attributes of the `transport` event type [_audit_event_attributes_of_the_transport_event_type] - -The events with `event.type` equal to `transport` have one of the following `event.action` attribute values: `authentication_success`, `anonymous_access_denied`, `authentication_failed`, `realm_authentication_failed`, `access_granted`, `access_denied`, `run_as_granted`, `run_as_denied`, or `tampered_request`. These events also have the following extra attributes (in addition to the common ones): - -`action` -: The name of the transport action that was executed. This is like the URL for a REST request. - -`indices` -: The indices names array that the request associated with this event pertains to (when applicable). - -`request.name` -: The name of the request handler that was executed. - - -## Audit event attributes of the `ip_filter` event type [_audit_event_attributes_of_the_ip_filter_event_type] - -The events with `event.type` equal to `ip_filter` have one of the following `event.action` attribute values: `connection_granted` or `connection_denied`. These events also have the following extra attributes (in addition to the common ones): - -`transport_profile` -: The transport profile the request targeted. - -`rule` -: The [IP filtering](../../security/ip-traffic-filtering.md) rule that denied the request. - - -## Audit event attributes of the `security_config_change` event type [security-config-change] - -The events with the `event.type` attribute equal to `security_config_change` have one of the following `event.action` attribute values: `put_user`, `change_password`, `put_role`, `put_role_mapping`, `change_enable_user`, `change_disable_user`, `put_privileges`, `create_apikey`, `delete_user`, `delete_role`, `delete_role_mapping`, `invalidate_apikeys`, `delete_privileges`, `change_apikey`, or `change_apikeys`. - -These events also have **one** of the following extra attributes (in addition to the common ones), which is specific to the `event.type` attribute. The attribute’s value is a nested JSON object: - -`put` -: The object representation of the security config that is being created, or the overwrite of an existing config. It contains the config for a `user`, `role`, `role_mapping`, or for application `privileges`. - -`delete` -: The object representation of the security config that is being deleted. It can be the config for a `user`, `role`, `role_mapping` or for application `privileges`. - -`change` -: The object representation of the security config that is being changed. It can be the `password`, `enable` or `disable`, config object for native or built-in users. If an API key is updated, the config object will be an `apikey`. - -`create` -: The object representation of the new security config that is being created. This is currently only used for API keys auditing. If the API key is created using the [create API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) it only contains an `apikey` config object. If the API key is created using the [grant API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-grant-api-key.html) it also contains a `grant` config object. - -`invalidate` -: The object representation of the security configuration that is being invalidated. The only config that currently supports invalidation is `apikeys`, through the [invalidate API key API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-invalidate-api-key.html). - -The schemas of the security config objects mentioned above are as follows. They are very similar to the request bodies of the corresponding security APIs. - -`user` -: An object like: - - ```js - `{"name": , "enabled": , "roles": , - "full_name": , "email": , "has_password": , - "metadata": }`. - ``` - - The `full_name`, `email` and `metadata` fields are omitted if empty. - - -`role` -: An object like: - - ```js - `{"name": , "role_descriptor": {"cluster": , "global": - {"application":{"manage":{:}}}, "indices": [ {"names": , "privileges": , "field_security": - {"grant": , "except": }, "query": , - "allow_restricted_indices": }], "applications":[{"application": , - "privileges": , "resources": }], "run_as": , - "metadata": }}`. - ``` - - The `global`, `field_security`, `except`, `query`, `allow_restricted_indices` and `metadata` fields are omitted if empty. - - -`role_mapping` -: An object like: - - ```js - `{"name": , "roles": , "role_templates": [{"template": , - "format": }], "rules": , "enabled": , "metadata": }`. - ``` - - The `roles` and `role_templates` fields are omitted if empty. The `rules` object has a recursively nested schema, identical to what is passed in the [API request for mapping roles](../../users-roles/cluster-or-deployment-auth/mapping-users-groups-to-roles.md). - - -`privileges` -: An array of objects like: - - ```js - `{"application": , "name": , "actions": , - "metadata": }`. - ``` - - -`password` -: A simple object like: - - ```js - `{"user":{"name": }}` - ``` - - -`enable` -: A simple object like: - - ```js - `{"user":{"name": }}` - ``` - - -`disable` -: A simple object like: - - ```js - `{"user":{"name": }}` - ``` - - -`apikey` -: An object like: - - ```js - `{"id": , "name": , "expiration": , "role_descriptors": [], - "metadata": []}` - ``` - - The `role_descriptors` objects have the same schema as the `role_descriptor` object that is part of the above `role` config object. - - -The object for an API key update will differ in that it will not include a `name`. - -`grant` -: An object like: - - ```js - `{"type": , "user": {"name": , "has_password": }, - "has_access_token": }` - ``` - - -`apikeys` -: An object like: - - ```js - `{"ids": , "name": , "owned_by_authenticated_user": - , "user":{"name": , "realm": }}` - ``` - - The object for a bulk API key update will differ in that it will not include `name`, `owned_by_authenticated_user`, or `user`. Instead, it may include `metadata` and `role_descriptors`, which have the same schemas as the fields in the `apikey` config object above. - - -`service_token` -: An object like: - - ```js - `{"namespace":,"service":,"name":}` - ``` - - - -## Extra audit event attributes for specific events [_extra_audit_event_attributes_for_specific_events] - -There are a few events that have some more attributes in addition to those that have been previously described: - -* `authentication_success`: - - `realm` - : The name of the realm that successfully authenticated the user. If authenticated using an API key, this is the special value of `_es_api_key`. This is a shorthand attribute for the same information that is described by the `user.realm`, `user.run_by.realm` and `authentication.type` attributes. - - `user.name` - : The name of the *effective* user. This is usually the same as the *authenticated* user, but if using the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) this instead denotes the name of the *impersonated* user. If authenticated using an API key, this is the name of the API key owner. If authenticated using a service account token, this is the service account principal, i.e. `namespace/service_name`. - - `user.realm` - : Name of the realm to which the *effective* user belongs. If authenticated using an API key, this is the name of the realm to which the API key owner belongs. - - `user.run_by.name` - : This attribute is present only if the request is using the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) and denotes the name of the *authenticated* user, which is also known as the *impersonator*. - - `user.run_by.realm` - : Name of the realm to which the *authenticated* (*impersonator*) user belongs. This attribute is provided only if the request uses the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md). - - `authentication.type` - : Method used to authenticate the user. Possible values are `REALM`, `API_KEY`, `TOKEN`, `ANONYMOUS` or `INTERNAL`. - - `apikey.id` - : API key ID returned by the [create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) request. This attribute is only provided for authentication using an API key. - - `apikey.name` - : API key name provided in the [create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) request. This attribute is only provided for authentication using an API key. - - `authentication.token.name` - : Name of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. - - `authentication.token.type` - : Type of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. - -* `authentication_failed`: - - `user.name` - : The name of the user that failed authentication. If the request authentication token is invalid or unparsable, this information might be missing. - - `authentication.token.name` - : Name of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. If the request authentication token is invalid or unparsable, this information might be missing. - - `authentication.token.type` - : Type of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. If the request authentication token is invalid or unparsable, this information might be missing. - -* `realm_authentication_failed`: - - `user.name` - : The name of the user that failed authentication. - - `realm` - : The name of the realm that rejected this authentication. **This event is generated for each consulted realm in the chain.** - -* `run_as_denied` and `run_as_granted`: - - `user.roles` - : The role names as an array of the *authenticated* user which is being granted or denied the *impersonation* action. If authenticated as a [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md), this is always an empty array. - - `user.name` - : The name of the *authenticated* user which is being granted or denied the *impersonation* action. - - `user.realm` - : The realm name that the *authenticated* user belongs to. - - `user.run_as.name` - : The name of the user as which the *impersonation* action is granted or denied. - - `user.run_as.realm` - : The realm name of that the *impersonated* user belongs to. - -* `access_granted` and `access_denied`: - - `user.roles` - : The role names of the user as an array. If authenticated using an API key, this contains the role names of the API key owner. If authenticated as a [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md), this is always an empty array. - - `user.name` - : The name of the *effective* user. This is usually the same as the *authenticated* user, but if using the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) this instead denotes the name of the *impersonated* user. If authenticated using an API key, this is the name of the API key owner. - - `user.realm` - : Name of the realm to which the *effective* user belongs. If authenticated using an API key, this is the name of the realm to which the API key owner belongs. - - `user.run_by.name` - : This attribute is present only if the request is using the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) and denoted the name of the *authenticated* user, which is also known as the *impersonator*. - - `user.run_by.realm` - : This attribute is present only if the request is using the [run as authorization functionality](../../users-roles/cluster-or-deployment-auth/submitting-requests-on-behalf-of-other-users.md) and denotes the name of the realm that the *authenticated* (*impersonator*) user belongs to. - - `authentication.type` - : Method used to authenticate the user. Possible values are `REALM`, `API_KEY`, `TOKEN`, `ANONYMOUS` or `INTERNAL`. - - `apikey.id` - : API key ID returned by the [create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) request. This attribute is only provided for authentication using an API key. - - `apikey.name` - : API key name provided in the [create API key](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) request. This attribute is only provided for authentication using an API key. - - `authentication.token.name` - : Name of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. - - `authentication.token.type` - : Type of the [service account](../../users-roles/cluster-or-deployment-auth/service-accounts.md) token. This attribute is only provided for authentication using a service account token. diff --git a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md index c2ab49bcd4..5ae085e5fe 100644 --- a/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md +++ b/deploy-manage/monitor/logging-configuration/enabling-audit-logs.md @@ -21,13 +21,7 @@ Audit logs are only available on certain [subscription levels](https://www.elast You can log security-related events such as authentication failures and refused connections to monitor your cluster for suspicious activity (including data access authorization and user security configuration changes). Audit logging can be enabled independently for {{es}} and {{kib}}. -Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. - -::::{note} -Audit logs are disabled by default and must be explicitly enabled. -:::: - -This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK). +This section describes how to enable and configure audit logging in both {{es}} and {{kib}} for all supported deployment types, including self-managed clusters, {{ech}}, {{ece}} (ECE), and {{eck}} (ECK). ::::{important} In orchestrated deployments, audit logs must be shipped to a monitoring deployment; otherwise, they remain at container level and won't be accessible to users. For details on configuring log forwarding in orchestrated environments, refer to [logging configuration](../logging-configuration.md). @@ -35,12 +29,15 @@ In orchestrated deployments, audit logs must be shipped to a monitoring deployme When audit logging is enabled, security events are persisted to a dedicated `_audit.json` file on the host’s file system, on every cluster node. For more information, refer to [{{es}} logfile audit output](logfile-audit-output.md). -You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](./configuring-audit-logs.md). - ## Enable audit logging [enable-audit-logging-procedure] To enable {{es}} or {{kib}} audit logs, configure `xpack.security.audit.enabled` to `true` in **all {{es}} or {{kib}} nodes**, then restart the nodes to apply the changes. For detailed instructions, select your deployment type: +::::{note} +Audit logs are disabled by default and must be explicitly enabled. +:::: + + ::::::{tab-set} :::::{tab-item} Self-managed @@ -154,3 +151,6 @@ When enabled, audit logs are collected and shipped to the monitoring cluster ref :::::: +## Next steps + +You can configure additional options to control what events are logged and what information is included in the audit log. For more information, refer to [](./configuring-audit-logs.md). diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md index 857809855f..61a41f98ba 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-events-ignore-policies.md @@ -7,6 +7,7 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- # Elasticsearch audit events ignore policies [audit-log-ignore-policy] @@ -34,7 +35,7 @@ xpack.security.audit.logfile.events.ignore_filters: An audit event generated by the *kibana_system* user and operating over multiple indices , some of which do not match the indices wildcard, will not match. As expected, operations generated by all other users (even operating only on indices that match the *indices* filter) will not match this policy either. -Audit events of different types may have [different attributes](elasticsearch-audit-events.md#audit-event-attributes). If an event does not contain an attribute for which some policy defines filters, the event will not match the policy. For example, the following policy will never match `authentication_success` or `authentication_failed` events, irrespective of the user’s roles, because these event schemas do not contain the `role` attribute: +Audit events of different types may have [different attributes]() asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events/#audit-event-attributes. If an event does not contain an attribute for which some policy defines filters, the event will not match the policy. For example, the following policy will never match `authentication_success` or `authentication_failed` events, irrespective of the user’s roles, because these event schemas do not contain the `role` attribute: ```yaml xpack.security.audit.logfile.events.ignore_filters: diff --git a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md index b8e73169a1..dee12a2559 100644 --- a/deploy-manage/monitor/logging-configuration/logfile-audit-output.md +++ b/deploy-manage/monitor/logging-configuration/logfile-audit-output.md @@ -7,6 +7,7 @@ applies: ece: all eck: all stack: all + serverless: unavailable --- # Elasticsearch logfile audit output [audit-log-output] @@ -23,7 +24,7 @@ If you overwrite the `log4j2.properties` and do not specify appenders for any of For {{es}} configuration options that control event filtering in audit logs, refer to [](./configuring-audit-logs.md). -## Log entry format [audit-log-entry-format] +## Log entry format [audit-log-entry-format] The audit events are formatted as JSON documents, and each event is printed on a separate line in the `_audit.json` file. The entries themselves do not contain the end-of-line delimiter. The audit event JSON format is somewhat particular, as **most** fields follow a dotted name syntax, are ordered, and contain non-null string values. This format creates a structured columnar aspect, similar to a CSV, that can be more easily inspected visually (compared to an equivalent nested JSON document). @@ -31,4 +32,4 @@ There are however a few attributes that are exceptions to the above format. The When the `request.body` attribute is present (see [Auditing search queries](auditing-search-queries.md)), it contains a string value containing the full HTTP request body, escaped as per the JSON RFC 4677. -Refer to [audit event types](elasticsearch-audit-events.md) for a complete list of fields, as well as examples, for each entry type. \ No newline at end of file +Refer to [audit event types]() (asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events) for a complete list of fields, as well as examples, for each entry type. \ No newline at end of file diff --git a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md index 5c41bb75d3..ab8a988786 100644 --- a/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md +++ b/deploy-manage/monitor/logging-configuration/security-event-audit-logging.md @@ -16,6 +16,8 @@ Audit logging is a powerful feature that helps you monitor and track security-re Audit logging also provides forensic evidence in the event of an attack, and can be enabled independently for {{es}} and {{kib}}. +Use the {{kib}} audit logs in conjunction with {{es}} audit logging to get a holistic view of all security related events. {{kib}} defers to the {{es}} security model for authentication, data index authorization, and features that are driven by cluster-wide privileges. + In this section, you'll learn how to: * [](./enabling-audit-logs.md): Activate {{es}} or {{kib}} audit logs for all supported deployment types. @@ -27,3 +29,8 @@ In this section, you'll learn how to: * [Correlate audit events](./correlating-kibana-elasticsearch-audit-logs.md): Explore audit logs and understand how events from the same request are correlated. By following these guidelines, you can effectively audit system activity, enhance security monitoring, and meet compliance requirements. + +For a complete description of audit event details and format, refer to: + +* [Elasticsearch audit events]() asciidocalypse://elasticsearch/docs/reference/elasticsearch/elasticsearch-audit-events +* [Kibana audit events]() asciidocalypse://kibana/docs/reference/kibana-audit-events diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index c43ffea159..d59a3167c3 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -783,10 +783,9 @@ toc: - file: monitor/logging-configuration/configuring-audit-logs.md children: - file: monitor/logging-configuration/logfile-audit-events-ignore-policies.md - - file: monitor/logging-configuration/auditing-search-queries.md - file: monitor/logging-configuration/logfile-audit-output.md + - file: monitor/logging-configuration/auditing-search-queries.md - file: monitor/logging-configuration/correlating-kibana-elasticsearch-audit-logs.md - - file: monitor/logging-configuration/elasticsearch-audit-events.md - file: cloud-organization.md children: - file: cloud-organization/billing.md @@ -859,4 +858,4 @@ toc: - file: uninstall/uninstall-elastic-cloud-enterprise.md - file: uninstall/uninstall-elastic-cloud-on-kubernetes.md - file: uninstall/uninstall-a-self-managed-cluster.md - - file: uninstall/delete-a-cloud-deployment.md \ No newline at end of file + - file: uninstall/delete-a-cloud-deployment.md