[RFC] Revised index action authorization scheme

[nibix]

Introduction

This is a follow-up and extension of the RFC on Refined index authorization from #3905 and #5367.

Based on that, we already worked on an implementation, which current state can be found in #5399.

During implementation we found quite a few issues and oddities with the current index action authorization implementation. Many of these are documented inside the integration tests at https://github.com/opensearch-project/security/tree/main/src/integrationTest/java/org/opensearch/security/privileges/int_tests

The changes in #5399 have the goal to fix these. However, this sometimes requires quite fundamental, breaking changes.

This RFC shall document these changes and open a forum to discuss the approaches.

Feature flag

In order to keep backwards compatability, OpenSearch shall not use the new behavior by default initially. A feature flag will be provided to switch on the new behavior.

The feature flag will be named privileges_evaluation_type and located in the config.yml security plugin configuration under the dynamic attribute; it supports the values legacy (default) and next_gen. Setting it to next_gen enables the new functionality globally (note: I am not a big fan of this terminology, suggestions for improved terminology welcome!)

Example:

_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
     privileges_evaluation_type: next_gen

do_not_fail_on_forbidden mode

The do_not_fail_on_forbidden mode will be enabled by default in the next_gen mode. However, the mode will employ new, refined semantics. In some regards, these semantics are deviating quite significantly from the old mode.

The following semantics have changed:

• In the old implementation, the security plugin could split aliases or data streams into member indices in case there were only privileges for some of the member indices present. This had the effect that filtered aliases functionality could be silently deactivated because the security plugin would replace a filtered alias by one of its (unfiltered) member indices. The new mode will no longer try to split up datastreams or filtered aliases.
• In the old implementation, the "do_not_fail_on_forbidden" mode would be only applied for a predefined list of actions. In the new mode, it will be applied to all requests that implement IndicesRequest.Replaceable. Thus, also 3rd party plugins which are not known to the security plugin can use the functionality.
• In the old implementation, all requests were subject to the index reduction employed by the "do_not_fail_on_forbidden" mode. The new mode gives a request-level fine-grained control over the behavior: Requests which enumerate concrete indices without wildcards (like index_a,index_b/_search) are no longer subject to the index reduction performed by the "do_not_fail_on_forbidden" mode. Rather, if a privilege is missing to execute the request, the whole request will be denied. However, the user has the option to specifiy index_a,index_b/_search?ignore_unavailable=true in order to make sure that the request does not fail in case indices without privileges are present. If an index expression utilized wildcards, the "do_not_fail_on_forbidden" mode will be always active in the sense that the wildcard will only match indices with privileges.
• System indices and protected indices now integrate with the "do_not_fail_on_forbidden" mode; so, any index for which such a special protection applies will be automatically filtered from the request.

Alias privilege handling

Privilege handling for aliases has been refined in order to fix logic issues with alias specific actions.

• The indices:admin/aliases must be now explicitly given to any alias name this action shall be applied to.
• The same holds for any other action which can implicitly create an alias, like indices:admin/create with an alias option.
• In the old mode, reading from an alias or datastream was also possible if a user just had privileges for all members of the alias or datastream, but not for the alias or datastream itself. In the new mode, the user must have privileges on the alias or datastream itself in order to access it directly. Access via backing indices remains possible, though.

Analyze action without index

The _analyze action can be used with and without indices. If it is used without indices, the user now needs a privilege for indices:admin/analyze on an arbitrary index.

Point in time actions

The point in time actions /_cat/pit_segments and /_search/point_in_time have a special _all mode that returns segments for all open point in time sessions. To be able to use these, you now need the dedicated clusters privileges indices:data/read/point_in_time/readall or cluster:monitor/point_in_time/segments/_all, respectively.

Discontinued configuration options

When the new mode is active, a number of configuration options are no longer respected. This has several sources of motivation:

• The configuration option was originally introduced to provide backwards compatability and to avoid a breaking change. As we have a breaking change here anyway, it is fine to abandon the configuration option.
• The configuration option is no longer necessary with the new concepts.
• Reducing code and test complexity; currently, the authz part of the security plugin just has too many configuration options to make it properly maintaineable. See also [RFC] Healthy amount of configuration options #5624

The dropped config options are documented in the following sections:

config.yml

• filtered_alias_mode: As the new mode no longer splits up aliases, there is no need to warn about the use of filtered aliases. This flag is undocumented.
• respect_request_indices_options: As the index resolution is now done in core, this flag has no relevance any more. This flag is undocumented.
• do_not_fail_on_forbidden_empty: This flag has been provided for keeping backwards compatibility.

opensearch.yml

• plugins.security.filter_securityindex_from_all_requests: The security plugin will always filter the security index from requests
• plugins.security.system_indices.enabled: System index privilege handling will be always enabled
• plugins.security.system_indices.permission.enabled: System index permissions handling will be always enabled
• plugins.security.unsupported.restore.securityindex.enabled: This flag is undocumented and unsupported
• plugins.security.enable_snapshot_restore_privilege: This is replaced by the general purpose config option plugins.security.privileges_evaluation.actions.universally_denied_actions.
• plugins.security.check_snapshot_restore_write_privileges: This option also exists for backwards compatiblity. In the new mode, write privileges will be always checked for restore operations.

Additional new configuration attributes

Besides the already mentioned feature flag, we introduce a few config options which provide us more flexibility to influence privileges evaluator behavior. This is especially useful when mitigating bugs. These options should be considered as advanced options which are usually not used by normal users.

• plugins.security.privileges_evaluation.actions.force_as_cluster_actions: Allows to treat actions that are usually considered index privileges, explicitly as cluster privileges instead.
• plugins.security.privileges_evaluation.actions.universally_denied_actions: Denies all requests of normal users for these actions. Only super admins can use these actions.
• plugins.security.privileges_evaluation.actions.map_action_names: Allows remapping of action names to privilege names.