[Security Solution][Endpoint] Update response action creation so that it store policy information when action is created #218175

paul-tavares · 2025-04-14T20:39:01Z

Summary

The following changes are in support of space awareness for response actions (currently behind feature flag: endpointManagementSpaceAwarenessEnabled). All response actions will now start to store agent policy information.

A new property will be stored in the Action Request document that captures policy information about each agent the action was sent to (agent.policy: []).
All response action client instances must now be initiated with a spaceId
A new method was added to the internal Fleet Services to retrieve all namespace's in use by a given integration type in the active space
The SentinelOne and Microsoft run host scripts were enhaced so that the host VM name includes the space id

Testing space awareness setup

Warning

Microsoft Defender response actions will fail with a message indicating that it was unable to find elastic agens for the MS agent ids provided on input. This error will persist until a new build of Elsaticsearch is released. A change was needed to the kibana_system role

Enable feature flags

xpack.securitySolution.enableExperimental:
  - endpointManagementSpaceAwarenessEnabled

xpack.fleet.enableExperimental:
  - useSpaceAwareness

Switch Fleet to Space aware

POST /internal/fleet/enable_space_awareness
Elastic-Api-Version: 1,

Add additional mappings for response actions

Note

Only needed until the Endpoint Package is updated with these mappings

Ensure that the Endpoint package is installed

Best way to do this is to create at least one integration policy in fleet.

Add mappings if not already included in index mapping

Tip

These instructions can be done from the Kibana Dev tools console

Ensure index exists:

PUT /_data_stream/.logs-endpoint.actions-default

Ignore any resource_already_exists_exception error that might be returned.

Check to see if the mapping is already present:

GET /.logs-endpoint.actions-default/_mapping/field/agent.policy*

If the response returns an empty mappings, then mappings need to be added

Add mappings to the index:

PUT /.logs-endpoint.actions-default/_mapping
{
  "properties": {
    "agent": {
      "properties": {
        "policy": {
          "properties": {
            "agentId": { "type": "keyword" },
            "elasticAgentId": { "type": "keyword" },
            "integrationPolicyId": { "type": "keyword" },
            "agentPolicyId": { "type": "keyword" }
          }
        }
      }
    }
  }
}

Add data

You should now be all setup to load data for testing. Remember that space data visibility is mostly driven by Fleet and how Policies are setup and shared between spaces - Agent Policies now have a Space ID field to manage this.

Our scripts that run live VMs for each type o9f EDR have been updated to support a --spaceId CLI argument and thus can be used to target specific spaces.

Checklist

Unit or functional tests were updated or added to match the most common scenarios

…subclasses

…ng the `agent.policy` info only if FF is enabled

…e-policy-id-in-action-request-for-spaces

This reverts commit 7366645.

… Endpoint to use it when returning agent info.

unable to test it at this time - role needs additional privileges

… VM name

… against specific indexes rather than index patterns

paul-tavares · 2025-04-14T20:39:52Z

/ci

paul-tavares · 2025-04-14T22:05:26Z

/ci

elasticmachine · 2025-04-17T14:33:19Z

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

…gating to core server

…n-action-request-for-spaces' into task/olm-11225-store-policy-id-in-action-request-for-spaces

…uest-for-spaces

szwarckonrad

Code and logic behind it LGTM!

szwarckonrad · 2025-04-22T11:20:46Z

...ions/security/plugins/security_solution/common/endpoint/types/microsoft_defender_endpoint.ts

+  };
+  microsoft: {
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    defender_endpoint: Record<string, any>;


Maybe we can go with an unknown?

I'll check if unknown will work as well.

x-pack/solutions/security/plugins/security_solution/server/endpoint/mocks/utils.mock.ts

...tion/server/endpoint/services/actions/clients/crowdstrike/crowdstrike_actions_client.test.ts

...curity_solution/server/endpoint/services/actions/clients/endpoint/endpoint_actions_client.ts

...solution/server/endpoint/services/actions/clients/sentinelone/sentinel_one_actions_client.ts

szwarckonrad · 2025-04-22T12:34:00Z

...solution/server/endpoint/services/actions/clients/sentinelone/sentinel_one_actions_client.ts

    connectorActions.setup(SENTINELONE_CONNECTOR_ID);
  }

+  private async fetchSentinelOneAgentIndexNames(): Promise<string[]> {


Feels like this could be abstracted in the base class — just caching and retrieving, right? I think you're already doing something similar for other connectors. Ignore this if that’s not the case.

I usually push methods/logic to the base class only if its something that applies to all EDR types (generic). In this case, the index names will be different depending on the EDR type, so I felt like it was most appropriate for it to live in each respective sub-class.

From konrad Co-authored-by: Konrad Szwarc <[email protected]>

…e-policy-id-in-action-request-for-spaces

…uest-for-spaces

ashokaditya · 2025-04-22T13:45:37Z

x-pack/solutions/security/plugins/security_solution/server/endpoint/routes/error_handler.ts


-  // Kibana CORE will take care of `500` errors when the handler `throw`'s, including logging the error
-  throw error;
+  // Kibana core server handling of `500` errors does not actually return the `error.message` encountered,


ashokaditya · 2025-04-22T13:56:43Z

...curity_solution/server/endpoint/services/actions/clients/lib/base_response_actions_client.ts

    const doc: LogsEndpointAction<TParameters, TOutputContent, TMeta> = {
      '@timestamp': new Date().toISOString(),
+      // Need to suppress this TS error around `agent.policy` not supporting `undefined`.
+      // It will be removed once we enable the feature and delete the feature flag checks.


Maybe add a // TODO here for visibility

ashokaditya · 2025-04-22T14:11:32Z

...ons/security/plugins/security_solution/common/endpoint/service/response_actions/constants.ts

  ],
  crowdstrike: ['device.id'],
  microsoft_defender_endpoint: [
+    'cloud.instance.id',


🙇🏼 Thanks for sorting!

😄

FYI:
I actually did not sort it on purpose. I only moved cloud.instance.id up so that it is the first one we try to find in MS documents. thats because I'm still not 100% sure of the other fields defined below because we never really set up the M365 integration. I got these values from an Env. that Raquel has where she did setup M365.

Ah I see. Well, sorted values are always nice to maintain. 🌞

...ions/security/plugins/security_solution/common/endpoint/types/microsoft_defender_endpoint.ts

x-pack/solutions/security/plugins/security_solution/common/endpoint/types/sentinel_one.ts

…uest-for-spaces

Co-authored-by: Ash <[email protected]>

…point/types/sentinel_one.ts

…uest-for-spaces

elasticmachine · 2025-04-23T18:22:26Z

💔 Build Failed

Buildkite Build
Commit: a07ed42

Failed CI Steps

Test Failures

[job] [logs] Jest Tests #4 / Update Insights Route Handler standard operations and authz should throw if retrieved insight is missing
[job] [logs] Jest Tests #4 / Update Insights Route Handler standard operations and authz should throw if retrieved insight is missing

Metrics [docs]

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`securitySolution`	85	86	+1

Total ESLint disabled count

id	before	after	diff
`securitySolution`	670	671	+1

History

💔 Build #295354 failed 28fcab4
💔 Build #295113 failed e029581
💔 Build #294988 failed d1b6b15
💔 Build #294915 failed 47daa45
💚 Build #294615 succeeded df5507e

cc @paul-tavares

… it store policy information when action is created (elastic#218175) ## Summary The following changes are in support of space awareness for response actions (currently behind feature flag: `endpointManagementSpaceAwarenessEnabled`). All response actions will now start to store agent policy information. - A new property will be stored in the Action Request document that captures policy information about each agent the action was sent to (`agent.policy: []`). - All response action client instances must now be initiated with a `spaceId` - A new method was added to the internal Fleet Services to retrieve all `namespace`'s in use by a given integration type in the active space - The SentinelOne and Microsoft run host scripts were enhaced so that the host VM name includes the space id

paul-tavares added 14 commits April 4, 2025 14:20

Added new abstract method fetchAgentPolicyInfo() to base class and …

f30a292

…subclasses

Added spaceId option to Response Actions clients and adjusted stori…

052bb8b

…ng the `agent.policy` info only if FF is enabled

Fleet: add log when running external callback fails

7366645

Merge remote-tracking branch 'upstream/main' into task/olm-11225-stor…

002f702

…e-policy-id-in-action-request-for-spaces

Revert "Fleet: add log when running external callback fails"

b3482d0

This reverts commit 7366645.

Add fetchAgentPolicyInfo() to Endpoint response actions client

b70f3dc

Implementation of fetchAgentPolicyInfo() for S1

6b5f953

remove comment

8e18f9e

Added fetchFleetInfoForAgents() to base class and refactored S1 and…

f597e67

… Endpoint to use it when returning agent info.

Added fetchAgentPolicyInfo() implementation to MS defender client

3444532

unable to test it at this time - role needs additional privileges

Added implementation of fetchAgentPolicyInfo() to crowdstrike

e2f3b37

Changed S1 and MS Defender run scripts to include the space id in the…

a06b59f

… VM name

Added new method getIntegrationNamespaces() to internal fleet services

62746f7

Refactor response actions clients to ensure policy info. is retrieved…

46ad24f

… against specific indexes rather than index patterns

paul-tavares added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v9.1.0 labels Apr 14, 2025

paul-tavares self-assigned this Apr 14, 2025

Fix generator and tests to include new agent.policy

f9e1fe1

paul-tavares added 8 commits April 15, 2025 09:33

Tests for getIntegrationNamespaces()

04e0198

Fix test for fetchActionRequests()

d710fba

Test for base class write of action to index

151ff88

Added additional mock generators for sentinelOne

8b3865b

Added support for index patterns in applyEsClientSearchMock() utility

7f9ff2d

Update mocks

5ae36a5

tests for S1 when spaces is enabled

48f024d

tests for endpoint write of request when spaces is enabled

9c62359

paul-tavares requested review from ashokaditya and pzl April 17, 2025 14:33

paul-tavares added 5 commits April 17, 2025 11:20

Change API route error handler to handle 500 errors instead of dele…

af22a7a

…gating to core server

Add additional validation to external EDR clients

7d36a91

Merge remote-tracking branch 'origin/task/olm-11225-store-policy-id-i…

317971a

…n-action-request-for-spaces' into task/olm-11225-store-policy-id-in-action-request-for-spaces

Fix in S1

08c7630

Fix test

6a3d122

paul-tavares requested review from szwarckonrad and removed request for pzl April 17, 2025 20:05

paul-tavares added a commit to paul-tavares/kibana that referenced this pull request Apr 17, 2025

-----[ PR elastic#218175 DROP ON REBASE ]------

cba7e93

Merge branch 'main' into task/olm-11225-store-policy-id-in-action-req…

df5507e

…uest-for-spaces

szwarckonrad approved these changes Apr 22, 2025

View reviewed changes

paul-tavares and others added 5 commits April 22, 2025 08:45

Apply suggestions from code review

a90a129

From konrad Co-authored-by: Konrad Szwarc <[email protected]>

Merge remote-tracking branch 'upstream/main' into task/olm-11225-stor…

5241583

…e-policy-id-in-action-request-for-spaces

Update type to use unknown

47daa45

Fix spelling and tests

d1b6b15

Merge branch 'main' into task/olm-11225-store-policy-id-in-action-req…

e029581

…uest-for-spaces

ashokaditya approved these changes Apr 23, 2025

View reviewed changes

paul-tavares and others added 3 commits April 23, 2025 08:38

Merge branch 'main' into task/olm-11225-store-policy-id-in-action-req…

057068f

…uest-for-spaces

Apply suggestions from code review

6f5407e

Co-authored-by: Ash <[email protected]>

Update x-pack/solutions/security/plugins/security_solution/common/end…

28fcab4

…point/types/sentinel_one.ts

paul-tavares enabled auto-merge (squash) April 23, 2025 12:44

Merge branch 'main' into task/olm-11225-store-policy-id-in-action-req…

a07ed42

…uest-for-spaces

Fix insights tests

76f9d08

paul-tavares merged commit 70d6f40 into elastic:main Apr 23, 2025
9 checks passed

paul-tavares deleted the task/olm-11225-store-policy-id-in-action-request-for-spaces branch April 23, 2025 21:09

ashokaditya mentioned this pull request Sep 12, 2025

[EDR Workflows] Update fetchAgentPolicyInfo query in CrowdStrike #234772

Merged

[Security Solution][Endpoint] Update response action creation so that it store policy information when action is created #218175

[Security Solution][Endpoint] Update response action creation so that it store policy information when action is created #218175

Uh oh!

Conversation

paul-tavares commented Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing space awareness setup

Enable feature flags

Switch Fleet to Space aware

Add additional mappings for response actions

Ensure that the Endpoint package is installed

Add mappings if not already included in index mapping

Add data

Checklist

Uh oh!

paul-tavares commented Apr 14, 2025

Uh oh!

paul-tavares commented Apr 14, 2025

Uh oh!

elasticmachine commented Apr 17, 2025

Uh oh!

szwarckonrad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

ESLint disabled in files

Total ESLint disabled count

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

paul-tavares commented Apr 14, 2025 •

edited

Loading

elasticmachine commented Apr 23, 2025 •

edited

Loading