+ * In order to work across multiple Java versions, this project uses multi-release jars via the {@code mrjar} plugin, which makes it is + * possible to specify classes for specific Java versions in specific {@code src} folders (e.g. {@code main23} for classes available to + * Java 23+). + * All the versioned Java classes are merged into the bridge jar. Therefore, we must prefix the class name + * with the version, e.g. {@code Java23EntitlementCheckerHandle} and {@code Java23EntitlementChecker}. + *
+ */ +package org.elasticsearch.entitlement.bridge; diff --git a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java index 6db27fa19e8f6..2ea1012158c00 100644 --- a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java +++ b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/initialization/EntitlementInitialization.java @@ -101,7 +101,42 @@ public static EntitlementChecker checker() { return manager; } - // Note: referenced by agent reflectively + /** + * Initializes the Entitlement system: + *+ * The third step is needed as the JDK exposes some API through interfaces that have different (internal) implementations + * depending on the JVM host platform. As we cannot instrument an interfaces, we find its concrete implementation. + * A prime example is {@link FileSystemProvider}, which has different implementations (e.g. {@code UnixFileSystemProvider} or + * {@code WindowsFileSystemProvider}). At runtime, we find the implementation class which is currently used by the JVM, and add + * its methods to the set of methods to instrument. See e.g. {@link EntitlementInitialization#fileSystemProviderChecks}. + *
+ *+ * NOTE: this method is referenced by the agent reflectively + *
+ * + * @param inst the JVM instrumentation class instance + */ public static void initialize(Instrumentation inst) throws Exception { manager = initChecker(); diff --git a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/instrumentation/InstrumentationService.java b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/instrumentation/InstrumentationService.java index ece51a8414b70..48e7288c19d82 100644 --- a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/instrumentation/InstrumentationService.java +++ b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/instrumentation/InstrumentationService.java @@ -22,6 +22,32 @@ record InstrumentationInfo(MethodKey targetMethod, CheckMethod checkMethod) {} Instrumenter newInstrumenter(Class clazz, Map+ * NOTE: look up of methods using this convention is the primary way we use to identify which methods to instrument, + * but other methods can be added to the map of methods to instrument. See + * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization#initialize} for details. + *
+ * + * @param clazz the class to inspect to find methods to instrument + * @throws ClassNotFoundException if the class is not defined or cannot be inspected + */ Map+ * Policies for ES plugins and modules are stored in a YAML file bundled with the module/plugin + * (@see README.md for details). The files are + * extracted from the bundles and parsed during Elasticsearch initialization ({@code Elasticsearch#initPhase2}); at the same time, we parse + * both plugin and server policy patches from the command line. Patches to plugin policies are applied immediately, the server policy patch + * is passed down to {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap}. + *
+ *+ * The server and agent (APM) policies are created in EntitlementInitialization, just before the creation of + * {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager}. The server policy patch (if any) is read from the + * {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap} arguments and applied here. + *
+ *+ * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} creates + * {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager} passing down the policies it just created (server and agent) and the + * plugin policies it read from {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap}. + *
+ * + *+ * Instrumentation happens dynamically via a Java Agent ({@code EntitlementAgent}, see the {@code agent} subproject). + *
+ *+ * The Agent is loaded dynamically by {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap}. We load the agent dynamically + * because we don't want to define additional permissions that server would need; we perform several sensitive actions once during + * Elasticsearch initialization. By initializing entitlements after we have performed those actions, we are able to never allow certain + * actions, like process execution. An additional benefit is that we are able to collect all the information needed before creating the + * entitlement objects, so they can be immutable. + *
+ *+ * {@code EntitlementAgent} creates {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} and calls + * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization#initialize(java.lang.instrument.Instrumentation)} + * on it, both by using reflection. Agents are loaded into the unnamed module, which makes module exports awkward. To work around this, + * we keep minimal code in the agent itself, and instead use reflection to call into this library. + * {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap} uses {@link java.lang.Module#addExports} to export + * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} to the agent and make it available. + *
+ *+ * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} loads an + * {@link org.elasticsearch.entitlement.instrumentation.InstrumentationService} instance. + * {@link org.elasticsearch.entitlement.instrumentation.InstrumentationService} is an interface that encapsulates all bytecode manipulation + * operations. We use SPI to load an implementation for it; currently, the implementation uses ASM, and it is located in the + * {@code asm-provider} subproject. + *
+ * + *+ * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} builds the set of methods to instrument using + * {@link org.elasticsearch.entitlement.instrumentation.InstrumentationService#lookupMethods} on the version-specific subclass of + * {@link org.elasticsearch.entitlement.bridge.EntitlementChecker}. + * {@link org.elasticsearch.entitlement.bridge.EntitlementChecker} is the interface that contains the definition of all the check methods; + * it needs to be accessible by both this project and the code injected by the agent, therefore is located in a small, self-contained + * library ({@see the {@code bridge} subproject}). + *
+ *+ * See {@link org.elasticsearch.entitlement.instrumentation.InstrumentationService#lookupMethods} for details. + *
+ * + *+ * The {@code bridge} subproject uses multi-release jars via the {@code mrjar} plugin, which makes it is possible to specify classes for + * specific Java versions in specific {@code src} folders (e.g. {@code main23} for classes available to Java 23+). + *
+ *+ * At runtime, we identify and instantiate the correct class using the runtime Java version to prepend the correct prefix to the class + * name, e.g. {@code Java21EntitlementChecker} for Java version 21 (see {@code EntitlementInitialization#getVersionSpecificCheckerClass}). + *
+ *+ * These different classes are needed to hold entitlements check definitions that are specific to a Java version. + * As an example, consider the Linker API. + *
+ *+ * Note: the current version of Elasticsearch supports Java 21+; this is only an example (taken from the 8.x branch) that + * illustrates a complex scenario. + *
+ *+ * The API went through multiple previews, and therefore changes between Java 19, 20 and 21; in order to support this correctly on these + * versions, we should introduce 2 utility interfaces, "preview" and "stable". + * For example, for the Java 20 specific signatures and functions, we would create {@code Java20StableEntitlementChecker} and + * {@code Java20PreviewEntitlementChecker}. + *
+ *+ * The linker API in Java 20 introduces the final form for {@code downcallHandle}, which has different argument types from the one in + * Java 19. To instrument and check it, we would add a + * {@code check$jdk_internal_foreign_abi_AbstractLinker$downcallHandle(FunctionDescriptor, Linker.Option...)} method for it to the + * {@code Java20StableEntitlementChecker} interface, which extends {@link org.elasticsearch.entitlement.bridge.EntitlementChecker}. + * This interface would then be used by both the Java 20 specific interface ({@code Java20EntitlementChecker}) and any interface for newer + * Java versions (e.g. {@code Java21EntitlementChecker}, which extends {@code Java20StableEntitlementChecker}): this way when we run on + * either Java 20, Java 21, or following versions, we always instrument {@code downcallHandle} with the Java 20+ signature defined in + * {@code Java20StableEntitlementChecker}. + * Java 20 also introduces the {@code upcallStub} function; this function is not in its final form, as it has different parameters in the + * following (21+) previews and in the final API. + * In this case, we would add a {@code jdk_internal_foreign_abi_AbstractLinker$upcallStub(MethodHandle, FunctionDescriptor, SegmentScope)} + * function to the {@code Java20PreviewEntitlementChecker} interface. {@code Java20EntitlementChecker} would inherit from this interface + * too, but {@code Java21EntitlementChecker} and following would not. This way when we run on Java 20 we would instrument {@code upcallStub} + * with the Java 20 signature {@code (FunctionDescriptor, Linker.Option...)}, but we would not when we run on following (Java 21+) versions. + * Those will have the newer (final) {@code upcallStub} definition introduced in {@code Java21EntitlementChecker}. + *
+ * + *+ * Agents get access to the Java instrumentation API by receiving a {@link java.lang.instrument.Instrumentation} instance, which we pass + * to {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization#initialize} to setup code needed to transform classes + * as they get loaded. See {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization#initialize} for details. + *
+ *+ * Our implementation instrument classes by adding a prologue to the methods identified in + * {@link org.elasticsearch.entitlement.initialization.EntitlementInitialization} (see previous section). + * See {@link org.elasticsearch.entitlement.instrumentation.Instrumenter#instrumentClass} for details. + *
+ * + *+ * In order to verify if a method is entitled to perform an action, we need to identify the right policy to check; the first step here is + * to identify the caller. This is done in the injected prologue, via a helper function + * {@link org.elasticsearch.entitlement.bridge.Util#getCallerClass}, which performs a limited stack walk. + *
+ * + *+ * The first step to find the set of entitlements granted to the caller class is to find the "layer" that hosts the class/module. + * Each layer may have a policy attached to it (1-1* relationship). + *
+ *+ * This starts during Elasticsearch initialization ({@code initPhase2}), just after policies are parsed but before entitlements are + * initialized via {@link org.elasticsearch.entitlement.bootstrap.EntitlementBootstrap}, through a new class named {@code PluginsLoader}. + * Before 8.18, {@code PluginsServices} (created as part of {@code Node} initialization in {@code initPhase3}) had 2 concerns: + * create the "infrastructure" to load an ES plugin (or ES module), e.g. the module layer and class loader, and actually load the main + * plugin class and create an instance of it for the plugin. + * Now the first concern (create the module layer and class loader) has been refactored and moved to {@code PluginsLoader}, so it can + * happen separately and earlier, in Phase 2, before entitlements are initialized. + *
+ *+ * The module layers and class loaders are used to map a class to a layer (via the {@code PluginsResolver} class): we use them to build a + * Module -> Plugin name (String) map. For modularized plugins we use the list of modules defined in the module layer; for the + * non-modularized ones, we use the unnamed module which is unique to each plugin classloader. + *
+ *+ * This map is then passed down and stored by {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager}. Alongside this map, + * {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager} builds a set of references to modules + * that belong to what we call the "system layer", i.e. the layer containing what we consider system modules, and the set of modules + * that we consider belonging to the "server layer". + * {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager} uses this info to identify the layer, and therefore the policy and + * entitlements, for the caller class. + *
+ *+ * See {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager} for details. + *
+ * + *+ * The injected prologue calls a {@code check$} method on {@link org.elasticsearch.entitlement.bridge.EntitlementChecker}; its + * implementation (normally on {@link org.elasticsearch.entitlement.runtime.api.ElasticsearchEntitlementChecker}, unless it is a + * version-specific method) calls the appropriate methods on {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager}, + * forwarding the caller class and a specific set of arguments. These methods all start with check, roughly matching an entitlement type + * (e.g. {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager#checkInboundNetworkAccess}, + * {@link org.elasticsearch.entitlement.runtime.policy.PolicyManager#checkFileRead}). + *
+ *+ * Most of the entitlements are "flag" entitlements: when present, it grants the caller the right to perform an action (or a set of + * actions); when it's not present, the actions associated with it are denied. Checking is simply a fact checking if the entitlement type + * is present or not. + *
+ * There are two entitlements that are not simple flags: + *+ * A final special cases that short circuit the checks (resulting in a "trivially allowed" case) is when the caller is null is the special + * {@code NO_CLASS} tag class - this happens if there are no frames in the call stack, e.g. when a call originated directly from native + * code (the JVM itself, a callback stub, a debugger, ...). + *
+ */ +package org.elasticsearch.entitlement; diff --git a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/runtime/policy/FileAccessTree.java b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/runtime/policy/FileAccessTree.java index cae5ddda6eb03..da1e42dd53c8a 100644 --- a/libs/entitlement/src/main/java/org/elasticsearch/entitlement/runtime/policy/FileAccessTree.java +++ b/libs/entitlement/src/main/java/org/elasticsearch/entitlement/runtime/policy/FileAccessTree.java @@ -36,6 +36,68 @@ import static org.elasticsearch.entitlement.runtime.policy.FileUtils.PATH_ORDER; import static org.elasticsearch.entitlement.runtime.policy.entitlements.FilesEntitlement.Mode.READ_WRITE; +/** + *+ * This structure facilitates looking up files entitlements for a particular component+module combination, given that grants can occur + * at the directory level in addition to the individual file level. + *
+ *+ * Broadly, this class operates on strings rather than abstractions like {@link java.io.File} or {@link Path}, since those can have + * behaviour that varies by platform in surprising ways; using strings makes the behaviour predictable. The strings are produced by a + * method called {@link FileAccessTree#normalizePath} to make sure they are absolute paths with consistent separator characters. + *
+ *+ * Internally, it does not use a tree data structure; the name "tree" refers to the tree structure of the filesystem, not the choice of + * data structure. It takes advantage of the fact that, after normalization, the name of a directory containing a file is a prefix of + * the file's own name. The permissions are maintained in several sorted arrays of {@link String}, and permission checks are implemented + * as a binary search within these arrays. + *
+ *+ * We want the binary search to locate the relevant entry immediately, either because there's an exact match, or else because there's no + * exact match and the binary search points us as an entry for the containing directory. This seems straightforward if the paths are + * absolute and sorted, but there are several subtleties. + *
+ *+ * Firstly, there could be intervening siblings; for example, in {@code ["/a", "/a/b"]}, if we look up {@code "/a/c"}, the binary search + * will land on {@code "/a/b"}, which is not a relevant entry for {@code "/a/c"}. The solution here is (1) segregate the read and write + * permissions so that each array covers one homogeneous kind of permission, and (2) prune the list so redundant child entries are removed + * when they are already covered by parent entries. In our example, this pruning process would produce the array {@code ["/a"]} only, + * and so the binary search for {@code "/a/c"} lands on the relevant entry for the containing directory. + *
+ *+ * Secondly, the path separator (whether slash or backslash) sorts after the dot character. This means, for example, if the array contains + * {@code ["/a", "/a.xml"]} and we look up {@code "/a/b"} using normal string comparison, the binary search would land on {@code "/a.xml"}, + * which is neither an exact match nor a containing directory, and so the lookup would incorrectly report no match, even though + * {@code "/a"} is in the array. To fix this, we define {@link FileUtils#PATH_ORDER} which sorts path separators before any other + * character. In the example, this would cause {@code "/a/b"} to sort between {@code "/a"} and {@code "/a.xml"} so that it correctly + * finds {@code "/a"}. + *
+ * With the paths pruned, sorted, and segregated by permission, each binary search has the following properties: + *+ * Some additional care is required in the unit tests for this code, since it must also run on Windows where the separator is a + * backslash and absolute paths don't start with a separator. See {@code FileAccessTreeTests#testDuplicateExclusivePaths} for an example. + *
+ */ public final class FileAccessTree { /** @@ -107,8 +169,18 @@ static void validateExclusivePaths(List+ * To find a component: + *
+ * When a check is performed (e.g. {@link PolicyManager#checkExitVM(Class)}, we get the module the caller class belongs to via + * {@link Class#getModule} and try (in order) to see if that class belongs to: + *
+ * Once it has a component, this class maps it to a policy and check the action performed by the caller class against its entitlements, + * either allowing it to proceed or raising a {@link NotEntitledException} if the caller class is not entitled to perform the action. + *
+ *+ * All these methods start in the same way: the components identified in the previous section are used to establish if and how to check: + * If the caller class belongs to {@link PolicyManager#SYSTEM_LAYER_MODULES}, no check is performed (the call is trivially allowed, see + * {@link PolicyManager#isTriviallyAllowed}). + * Otherwise, we lazily compute and create a {@link PolicyManager.ModuleEntitlements} record (see + * {@link PolicyManager#computeEntitlements}). The record is cached so it can be used in following checks, stored in a + * {@code Module -> ModuleEntitlement} map. + *
+ */ public class PolicyManager { /** * Use this if you don't have a {@link ModuleEntitlements} in hand. @@ -74,8 +136,23 @@ public class PolicyManager { static final Set+ * We use layers when computing {@link ModuleEntitlements}; first, we check whether the module we are building it for is in the + * server layer ({@link PolicyManager#SERVER_LAYER_MODULES}) (*). + * If it is, we use the server policy, using the same caller class module name as the scope, and read the entitlements for that scope. + * Otherwise, we use the {@code PluginResolver} to identify the correct plugin layer and find the policy for it (if any). + * If the plugin is modular, we again use the same caller class module name as the scope, and read the entitlements for that scope. + * If it's not, we use the single {@code ALL-UNNAMED} scope – in this case there is one scope and all entitlements apply + * to all the plugin code. + *
+ *+ * (*) implementation detail: this is currently done in an indirect way: we know the module is not in the system layer + * (otherwise the check would have been already trivially allowed), so we just check that the module is named, and it belongs to the + * boot {@link ModuleLayer}. We might want to change this in the future to make it more consistent/easier to maintain. + *
+ * + * @param componentName the plugin name or else one of the special component names like "(server)". */ record ModuleEntitlements( String componentName,