[Security Solution] Show only user-relevant events in the "Execution Events" tab

[nikitaindik]

Epic: https://github.com/elastic/security-team/issues/15617

Summary

This PR addresses some of the usability concerns for the "Execution events" tab on the rule details page. The tab is currently hidden behind a feature flag.

Goal of this PR is to avoid overburdening the user with the implementation details by default. However, writing debug and trace level events to event log can still be turned on using the securitySolution:extendedRuleExecutionLoggingMinLevel advanced setting.

Feature behaviour by default (info/warn/error levels)

• Show a single final rule execution status to the user. The final status contains warnings or errors if rule execution had any.
• Show a single "metrics" event at the end of rule execution

What this does NOT change

• All events at all levels are still written to the event log. Filtering is read-side only in the API.
• The "Execution results" tab and Detection Engine Health API are unaffected. They use their own aggregation queries.
• No schema changes to the event log.

What info-level events are shown to user

All of these are logged from a shared rule wrapper. These were added in a previous PR. Posting table here for context.

Event	Rule types	When shown
Found matching events: <N>	Query, EQL, ESQL, ML	Always (every execution)
Alerts created: <N>	All (incl. New Terms, Threshold)	Always (every execution)
Alerts suppressed: <N>	Query, EQL, ESQL (with suppression configured)	Only when suppressed count > 0
Events that did not result in alerts: <N>	Query, EQL, ESQL, Indicator Match, ML	Only when matched events > created + suppressed

Screenshots

🖼️ Click to see screenshots

Various rule execution statuses - only "Running" and a single final status are shown

Paging through ES – no flood of execution events

More screenshots

---

⚠️ Pragmatic stopgap: This PR takes a minimal-change approach to ship customer value now. A proper refactoring is planned for v9.5 in #16022 which will consolidate to a single structured event per execution. This PR is designed to be fully replaceable by that work — no new public APIs or schema changes.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[maximpn]

@nikitaindik Showing the latest rule execution status makes total sense to me 👍

I've taken the following screenshot locally. With the rule execution logging it's possible to track the important rule execution aspects

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: f5d23b3

Failed CI Steps

• Jest Integration Tests #3

Test Failures

• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) aggregations work without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) bulk operations work without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) cat APIs work without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) cluster health works without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) count works without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) index operations work without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) msearch works without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) PIT operations work without project_routing (PIT has its own scope)
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) scroll works without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS baseline: requests without project_routing (no-op cases) search works without project_routing
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing accepts project_routing parameter without error
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with _source filtering
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with aggregations
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with bool query
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with highlight
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with match query
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with pagination (from/size)
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with sort
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with term query
• [job] [logs] Jest Integration Tests #3 / project_routing on serverless CPS search API with project_routing works with wildcard index pattern

Metrics [docs]

✅ unchanged

History

• 💛 Build #400705 was flaky 027bc5b
• 💛 Build #400395 was flaky a6b2666

cc @nikitaindik

[banderror]

LGTM 👍