Add missing APM entitlements #123462

ldematte · 2025-02-26T08:43:48Z

We currently do not match these 3 permissions on elastic-apm-agent (module elastic.apm.agent)

permission java.net.SocketPermission "*", "connect,resolve";
permission org.elasticsearch.secure_sm.ThreadPermission "modifyArbitraryThreadGroup";
permission java.io.FilePermission "<<ALL FILES>>", "read,write";

I've added the first 2; I was surprised we did not caught the first one, it's pretty important to communicate to the APM agent, but then I looked at IT tests and we might have a miss there (we test only to localhost).
It's worth more investigation though, I made a note to do that.
The second one might be already covered by agentEntitlements, but I added it to the module as well - I think it's safer to err on the safer side :)

I'm very reluctant to add the third permission however. Read and write to all files? No. Wdyt?

Relates to ES-10031

Complete picture for reference:
(legend: x we have it, - we don't care about it, ? missing

grant {
 -   permission java.lang.RuntimePermission "accessSystemModules";
 x   permission java.lang.RuntimePermission "createClassLoader";
 -   permission java.lang.RuntimePermission "getClassLoader";
 x   permission java.util.PropertyPermission "elastic.apm.*", "write";
 x   permission java.util.PropertyPermission "*", "read,write";
 -   permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
};

grant codeBase "${codebase.elastic-apm-agent}" {
-    permission java.lang.RuntimePermission "accessDeclaredMembers";
-    permission java.lang.RuntimePermission "setContextClassLoader";
x    permission java.lang.RuntimePermission "setFactory";
-    permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
?    permission java.net.SocketPermission "*", "connect,resolve";
    // profiling function in APM agent
x    permission java.util.PropertyPermission "AsyncProfiler.safemode", "write";
-    permission java.lang.RuntimePermission "accessUserInformation";
x    permission java.lang.RuntimePermission "loadLibrary.*";
-    permission java.lang.RuntimePermission "getClassLoader";
?    permission java.io.FilePermission "<<ALL FILES>>", "read,write";
?    permission org.elasticsearch.secure_sm.ThreadPermission "modifyArbitraryThreadGroup";
-    permission java.net.NetPermission "getProxySelector";
};

elasticsearchmachine · 2025-02-26T08:44:12Z

Pinging @elastic/es-core-infra (Team:Core/Infra)

mosche · 2025-02-26T09:01:35Z

I checked in QA, it looks like we're in deed missing permissions for the agent to write it's log file. Though, it could also be due to the fact that we're just logging >= WARN. We should deploy a project with INFO logging for the agent to be sure.

mosche · 2025-02-26T09:04:15Z

I can see metrics, so it seems to be able to connect to APM server

mosche · 2025-02-26T09:50:52Z

modules/apm/src/main/plugin-metadata/entitlement-policy.yaml

        - AsyncProfiler.safemode
  - load_native_libraries
+  - manage_threads
+  - outbound_network


wondering, why is elastic.apm.agent needed at all? shouldn't this be covered by the default agent policy?
a bit worrying is that we receive metrics from my QA projects despite not allowing outbound_network here or in the default agent policy 😱

Were they apm metrics or just generic container metrics provided by metricbeat on the host?

The issue about missing outbound_network but still working might be due to missing instrumentation to URLConnection & derived that we discovered.

why is elastic.apm.agent needed at all

Hypothesis (that I will try to confirm): in the plugin that jar/module is used kind of a client/an API to talk to the agent/to the server?

But I do wonder if we need these policies in both places, or just here, or just on the agentEntitlements fixed set.
I'll dig a bit more on this point too.

I moved these entitlements to the agentEntitlements policy; considering the dependency is runtimeOnly and the code using these entitlements is in the agent, seems like that is the right place for them.
Tests pass, even though APM tests are really limited - I tried to run ES with-apm-server too and no errors there.
Will also try a Serverless QA deployment before merging.

…g-policies-3

mosche

lgtm

ldematte · 2025-02-28T18:22:25Z

...nt/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java

+            new LoadNativeLibrariesEntitlement(),
            new FilesEntitlement(
                List.of(
                    FileData.ofPath(Path.of("/co/elastic/apm/agent/"), READ),


Actually these weird paths are ZipPaths! #123744 will fix that and we can remove them.

ldematte · 2025-02-28T18:23:28Z

...nt/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java

            new SetHttpsConnectionPropertiesEntitlement(),
            new OutboundNetworkEntitlement(),
+            new WriteSystemPropertiesEntitlement(Set.of("AsyncProfiler.safemode")),
+            new LoadNativeLibrariesEntitlement(),


As @rjernst pointed out, we don't need this anymore: the APM Agent is loaded before entitlements are turned on, so native loading happens before that and an entitlement is not needed. I'll remove this.

…g-policies-3

elasticsearchmachine · 2025-03-04T07:34:29Z

💔 Backport failed

Status	Branch	Result
❌	8.18	Commit could not be cherrypicked due to conflicts
❌	8.x	Commit could not be cherrypicked due to conflicts
❌	9.0	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 123462

Add missing APM entitlements

a58dd34

ldematte added >non-issue auto-backport Automatically create backport pull requests when merged v8.18.1 v8.19.0 v9.0.1 v9.1.0 :Core/Infra/Entitlements Entitlements infrastructure labels Feb 26, 2025

elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Feb 26, 2025

ldematte requested a review from a team February 26, 2025 08:44

mosche reviewed Feb 26, 2025

View reviewed changes

ldematte mentioned this pull request Feb 27, 2025

Add missing APM agent entitlements #123575

Merged

ldematte added 3 commits February 27, 2025 12:10

Moving all APM agent permissions to the agent policy

4738240

Merge remote-tracking branch 'upstream/main' into entitlements/missin…

6828df0

…g-policies-3

Unmute APM tests

e5183d9

ldematte requested a review from mosche February 27, 2025 11:23

mosche approved these changes Feb 27, 2025

View reviewed changes

ldematte commented Feb 28, 2025

View reviewed changes

ldematte added 3 commits March 3, 2025 15:09

Merge remote-tracking branch 'upstream/main' into entitlements/missin…

c684a9d

…g-policies-3

removed jar/zip paths, added logs dir

80defc5

Merge branch 'main' into entitlements/missing-policies-3

56ecceb

ldematte merged commit c663e3d into elastic:main Mar 4, 2025
17 checks passed

ldematte deleted the entitlements/missing-policies-3 branch March 4, 2025 07:33

elasticsearchmachine added the backport pending label Mar 4, 2025

ldematte added a commit to ldematte/elasticsearch that referenced this pull request Mar 4, 2025

Add missing APM entitlements (elastic#123462)

98092fa

ldematte mentioned this pull request Mar 4, 2025

[9.0] Add missing APM entitlements (#123462) #123962

Merged

ldematte added a commit to ldematte/elasticsearch that referenced this pull request Mar 4, 2025

Add missing APM entitlements (elastic#123462)

ffa084d

ldematte mentioned this pull request Mar 4, 2025

[8.x] Add missing APM entitlements (#123462) #123963

Merged

ldematte added a commit to ldematte/elasticsearch that referenced this pull request Mar 4, 2025

Add missing APM entitlements (elastic#123462)

9f51113

ldematte mentioned this pull request Mar 4, 2025

[8.18] Add missing APM entitlements (#123462) #123964

Merged

ldematte removed the backport pending label Mar 4, 2025

elasticsearchmachine pushed a commit that referenced this pull request Mar 4, 2025

Add missing APM entitlements (#123462) (#123962)

f58d6ed

elasticsearchmachine pushed a commit that referenced this pull request Mar 4, 2025

Add missing APM entitlements (#123462) (#123964)

ac3742f

elasticsearchmachine pushed a commit that referenced this pull request Mar 4, 2025

Add missing APM entitlements (#123462) (#123963)

494ec9f

georgewallace pushed a commit to georgewallace/elasticsearch that referenced this pull request Mar 11, 2025

Add missing APM entitlements (elastic#123462)

dbf074a

Add missing APM entitlements #123462

Add missing APM entitlements #123462

Uh oh!

Conversation

ldematte commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Feb 26, 2025

Uh oh!

mosche commented Feb 26, 2025

Uh oh!

mosche commented Feb 26, 2025

Uh oh!

mosche Feb 26, 2025

Choose a reason for hiding this comment

Uh oh!

mark-vieira Feb 26, 2025

Choose a reason for hiding this comment

Uh oh!

ldematte Feb 26, 2025

Choose a reason for hiding this comment

Uh oh!

ldematte Feb 26, 2025

Choose a reason for hiding this comment

Uh oh!

ldematte Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

mosche left a comment

Choose a reason for hiding this comment

Uh oh!

ldematte Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

ldematte Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 4, 2025

💔 Backport failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ldematte commented Feb 26, 2025 •

edited

Loading