[Internal]: Add known issue to release notes for "Entity Risk Score documents eventually fail to persist in 8.18/9.0" #1548
Description
Description
In environments where the Security Entity Risk score feature was enabled prior to 8.18, and then after an upgrade to Elastic version 8.18 (or 9.0), the Security Entity risk score feature eventually stops persisting risk score documents.
This is due to a bug where an Ingest pipeline “entity_analytics_create_eventIngest_from_timestamp-pipeline-<space_name>” which is set as a default pipeline for the risk scoring index in 8.18, is not being created on Kibana startup. Immediately after the upgrade, the document persistence will likely succeed. However, it may take anywhere between 0 and 30 days before the documents begin failing to be written, as that is how long it takes for the Risk Score Data stream to roll over, and its underlying index’s settings to take on the new default pipeline.
This issue does not apply to customers who created an Elasticsearch cluster on 8.18/9.0 directly, nor does it apply to customers who didn’t enable Entity Risk scoring until 8.18/9.0. When installing the risk engine in those versions directly, the appropriate resources are created.
We should add a known issue about that to the following release notes: v8.18.x, v9.0.x, Serverless.
The bug is currently planned to be fixed in: v8.18.3, v9.0.3 as well as in Serverless as soon as possible.
Workaround
For customers already on 8.18/9.0, who are experiencing this issue, a workaround is to create the ingest pipeline manually in each space in which Entity Risk scoring is installed. This can be done via a single PUT request, as described below. Note that “default” in the example ingest pipeline name below is the space ID. Also note that customers may perform this workaround prior to the 8.18 upgrade as well to ensure no issues.
PUT /_ingest/pipeline/entity_analytics_create_eventIngest_from_timestamp-pipeline-default
{
"_meta": {
"managed_by": "entity_analytics",
"managed": true
},
"description": "Pipeline for adding timestamp value to event.ingested",
"processors": [
{
"set": {
"field": "event.ingested",
"value": "{{_ingest.timestamp}}"
}
}
]
}
After this step is complete, risk scores should automatically begin to persist successfully in the next run of the Entity Risk Engine. The next run time is described on the “Security / Entity Risk Score” management screen, and the next run can also be triggered manually by pressing the “Run Engine” button.
Resources
- KnowledgeBase article: https://support.elastic.dev/knowledge/view/08ed2d48
- Pull Request with the fix (currently in Draft State): Install event.ingested ingest pipeline in all spaces that the risk engine or asset criticality is installed, and add the pipeline to their indices kibana#221937
Which documentation set does this change impact?
Release notes for Kibana and Serverless
Feature differences
N/A
What release is this request related to?
N/A
Serverless release
N/A
Collaboration model
The documentation team
Point of contact.
Main contact: @jaredburgettelastic @hop-dev