Skip to content

Deprecating data_frame_transforms roles #117519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jan 6, 2025
Merged

Deprecating data_frame_transforms roles #117519

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1080975
Deprecating data_frame_transforms roles
dan-rubinstein Nov 22, 2024
dafc7dd
Update docs/changelog/117519.yaml
dan-rubinstein Nov 25, 2024
a3dc6b0
Update changelog
dan-rubinstein Nov 26, 2024
1da2ebc
Merge branch 'main' into deprecate-transform-roles
elasticmachine Dec 9, 2024
aa4c6e1
Removing deprecation warning
dan-rubinstein Dec 10, 2024
ffcf7ae
Cleaning up unused role retrieval function
dan-rubinstein Dec 10, 2024
620bd86
Merge branch 'main' into deprecate-transform-roles
n1v0lg Dec 18, 2024
6f30bf2
Update docs/changelog/117519.yaml
dan-rubinstein Dec 23, 2024
418c261
Update changelog with breaking change details
prwhelan Dec 23, 2024
1bcdd82
Merge branch 'main' into deprecate-transform-roles
elasticmachine Dec 23, 2024
713f206
Revert ClusterPrivilegeResolver
prwhelan Dec 26, 2024
4599256
Remove Deprecated Transform Role permissions
prwhelan Dec 26, 2024
f9e0704
Merge branch 'main' into deprecate-transform-roles
n1v0lg Dec 27, 2024
485f0da
Update docs/changelog/117519.yaml
prwhelan Dec 27, 2024
bb6e4f7
Revert "Remove Deprecated Transform Role permissions"
prwhelan Dec 27, 2024
f4bd073
Revert docs, update changelog
prwhelan Dec 27, 2024
a57695e
Revert privilege tests
prwhelan Dec 27, 2024
7f3cb27
Merge branch 'main' into deprecate-transform-roles
dan-rubinstein Jan 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/changelog/117519.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
pr: 117519
summary: Deprecating `data_frame_transforms` roles
area: Machine Learning
type: deprecation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want a >breaking label on the PR instead of a >deprecation label. That'll also update some of the contents of this changelog IIRC.

After updating the label, lets tweak the title and summary here and the PR title to "Removing data_frame_transforms roles".

issues: []
deprecation:
title: Deprecating `data_frame_transforms` roles
area: Transform
details: >-
This change removes the `data_frame_transforms_admin`/`data_frame_transforms_user` roles.
These roles have been replaced by the equivalent `transform_admin`/`transform_user` roles.
impact: >-
Usage of the `data_frame_transforms_admin`/`data_frame_transforms_user` roles will no
longer succeed. Users should instead use the equivalent `transform_admin`/`transform_user` roles.
2 changes: 1 addition & 1 deletion docs/reference/rest-api/security/bulk-create-roles.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ The result would then have the `errors` field set to `true` and hold the error f
"details": {
"my_admin_role": { <4>
"type": "action_request_validation_exception",
"reason": "Validation Failed: 1: unknown cluster privilege [bad_cluster_privilege]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,manage_data_stream_global_retention,monitor_data_stream_global_retention,none,cancel_task,cross_cluster_replication,cross_cluster_search,delegate_pki,grant_api_key,manage_autoscaling,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_search_application,manage_search_query_rules,manage_search_synonyms,manage_service_account,manage_token,manage_user_profile,monitor_connector,monitor_enrich,monitor_inference,monitor_ml,monitor_rollup,monitor_snapshot,monitor_stats,monitor_text_structure,monitor_watcher,post_behavioral_analytics_event,read_ccr,read_connector_secrets,read_fleet_secrets,read_ilm,read_pipeline,read_security,read_slm,transport_client,write_connector_secrets,write_fleet_secrets,create_snapshot,manage_behavioral_analytics,manage_ccr,manage_connector,manage_enrich,manage_ilm,manage_inference,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_data_frame_transforms,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_data_frame_transforms,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions;"
"reason": "Validation Failed: 1: unknown cluster privilege [bad_cluster_privilege]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,manage_data_stream_global_retention,monitor_data_stream_global_retention,none,cancel_task,cross_cluster_replication,cross_cluster_search,delegate_pki,grant_api_key,manage_autoscaling,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_search_application,manage_search_query_rules,manage_search_synonyms,manage_service_account,manage_token,manage_user_profile,monitor_connector,monitor_enrich,monitor_inference,monitor_ml,monitor_rollup,monitor_snapshot,monitor_stats,monitor_text_structure,monitor_watcher,post_behavioral_analytics_event,read_ccr,read_connector_secrets,read_fleet_secrets,read_ilm,read_pipeline,read_security,read_slm,transport_client,write_connector_secrets,write_fleet_secrets,create_snapshot,manage_behavioral_analytics,manage_ccr,manage_connector,manage_enrich,manage_ilm,manage_inference,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions;"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to revert this

}
}
}
Expand Down
2 changes: 0 additions & 2 deletions docs/reference/rest-api/security/get-builtin-privileges.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,6 @@ A successful call returns an object with "cluster", "index", and "remote_cluster
"manage_behavioral_analytics",
"manage_ccr",
"manage_connector",
"manage_data_frame_transforms",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see mention of removing these privileges in the breaking changes request -- we only mention roles. The manage_data_frame_transforms and monitor_data_frame_transforms privileges are indeed deprecated according to our docs (monitor_data_frame_transforms has already been removed from docs it seems), but we don't issue any deprecation warnings around these privileges (currently there's no clean, built-in support for that). These privileges can be assigned to user-defined roles and user-created API keys so they are not just scoped to the built-in roles we're removing here.

I think it's likely still a good idea to remove the privileges in 9.0 but I prefer that we do this in a separate PR and after making sure that it's worth the trouble.

@dan-rubinstein could you revert the changes around privileges in this PR? That'll also take care of some of the CI failures. I can open a follow up PR with the privileges removed -- since it requires a PR against the Serverless repo as well, it'll be easier for me to author both PRs than only one of them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can revert the changes for this file - to clarify though, would you prefer that we revert the change entirely, or would you prefer that we set the permissions as empty via Set.of()?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prwhelan let's revert it entirely -- I synced on this with the ES sec team before the holidays and the preference from our end would be to follow the deprecate, then remove approach, i.e., implement a deprecation warning for the privilege, then remove in a later version. This would delay the removal of the privilege since we'd first need a mechanism for issuing deprecation warnings for privileges (that's on the ES security team to implement).

Would the following be acceptable for the ML team? Remove the roles now, wait for ES security to implement a deprecation mechanism, deprecate the privilege, then remove it in some later ES version? This way things are less disruptive for any customers that may still be relying on the privilege.

Just a preference: if you have confidence that only very few (if any) customers still use this privilege, or feel strongly about making it non-functional sooner, I'm also good with making it a NOOP via Set.of().

Either way though, let's do whatever we do around the privilege in a separate PR to keep things focused.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, is there an issue we can track for deprecating and removing the privilege? I imagine we'll need to wait to the v10 upgrade to delete it, in which case I want to record it in some way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup -- I'll create Jira for deprecating this particular privilege and Slack it to you (it's a private link that external people won't have access to).

"manage_data_stream_global_retention",
"manage_enrich",
"manage_ilm",
Expand All @@ -104,7 +103,6 @@ A successful call returns an object with "cluster", "index", and "remote_cluster
"manage_watcher",
"monitor",
"monitor_connector",
"monitor_data_frame_transforms",
"monitor_data_stream_global_retention",
"monitor_enrich",
"monitor_inference",
Expand Down
12 changes: 0 additions & 12 deletions docs/reference/security/authorization/built-in-roles.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,6 @@ suitable for writing beats output to {es}.

--

[[built-in-roles-data-frame-transforms-admin]] `data_frame_transforms_admin` ::
Grants `manage_data_frame_transforms` cluster privileges, which enable you to
manage {transforms}. This role also includes all
{kibana-ref}/kibana-privileges.html[Kibana privileges] for the {ml-features}.
deprecated:[7.5.0,"Replaced by <<built-in-roles-transform-admin,`transform_admin`>>"].

[[built-in-roles-data-frame-transforms-user]] `data_frame_transforms_user` ::
Grants `monitor_data_frame_transforms` cluster privileges, which enable you to
use {transforms}. This role also includes all
{kibana-ref}/kibana-privileges.html[Kibana privileges] for the {ml-features}.
deprecated:[7.5.0,"Replaced by <<built-in-roles-transform-user,`transform_user`>>"].

[[built-in-roles-editor]] `editor` ::

Grants full access to all features in {kib} (including Solutions) and read-only access to data indices.
Expand Down
6 changes: 0 additions & 6 deletions docs/reference/security/authorization/privileges.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,6 @@ only on clusters that contain follower indices.
+
This privilege is not available in {serverless-full}.

`manage_data_frame_transforms`::
All operations related to managing {transforms}.
deprecated[7.5] Use `manage_transform` instead.
+
This privilege is not available in {serverless-full}.

`manage_data_stream_global_retention`::
This privilege has no effect.deprecated[8.16]

Expand Down
10 changes: 0 additions & 10 deletions .../java/org/elasticsearch/xpack/core/security/authz/privilege/ClusterPrivilegeResolver.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,10 +234,6 @@ public class ClusterPrivilegeResolver {
MONITOR_INFERENCE_PATTERN
);
public static final NamedClusterPrivilege MONITOR_ML = new ActionClusterPrivilege("monitor_ml", MONITOR_ML_PATTERN);
public static final NamedClusterPrivilege MONITOR_TRANSFORM_DEPRECATED = new ActionClusterPrivilege(
"monitor_data_frame_transforms",
MONITOR_TRANSFORM_PATTERN
);
public static final NamedClusterPrivilege MONITOR_TEXT_STRUCTURE = new ActionClusterPrivilege(
"monitor_text_structure",
MONITOR_TEXT_STRUCTURE_PATTERN
Expand All @@ -253,10 +249,6 @@ public class ClusterPrivilegeResolver {
public static final NamedClusterPrivilege MANAGE = new ActionClusterPrivilege("manage", ALL_CLUSTER_PATTERN, ALL_SECURITY_PATTERN);
public static final NamedClusterPrivilege MANAGE_INFERENCE = new ActionClusterPrivilege("manage_inference", MANAGE_INFERENCE_PATTERN);
public static final NamedClusterPrivilege MANAGE_ML = new ActionClusterPrivilege("manage_ml", MANAGE_ML_PATTERN);
public static final NamedClusterPrivilege MANAGE_TRANSFORM_DEPRECATED = new ActionClusterPrivilege(
"manage_data_frame_transforms",
MANAGE_TRANSFORM_PATTERN
);
public static final NamedClusterPrivilege MANAGE_TRANSFORM = new ActionClusterPrivilege("manage_transform", MANAGE_TRANSFORM_PATTERN);
public static final NamedClusterPrivilege MANAGE_TOKEN = new ActionClusterPrivilege("manage_token", MANAGE_TOKEN_PATTERN);
public static final NamedClusterPrivilege MANAGE_WATCHER = new ActionClusterPrivilege("manage_watcher", MANAGE_WATCHER_PATTERN);
Expand Down Expand Up @@ -426,7 +418,6 @@ public class ClusterPrivilegeResolver {
MONITOR_INFERENCE,
MONITOR_ML,
MONITOR_TEXT_STRUCTURE,
MONITOR_TRANSFORM_DEPRECATED,
MONITOR_TRANSFORM,
MONITOR_WATCHER,
MONITOR_ROLLUP,
Expand All @@ -436,7 +427,6 @@ public class ClusterPrivilegeResolver {
MANAGE_CONNECTOR,
MANAGE_INFERENCE,
MANAGE_ML,
MANAGE_TRANSFORM_DEPRECATED,
MANAGE_TRANSFORM,
MANAGE_TOKEN,
MANAGE_WATCHER,
Expand Down
64 changes: 0 additions & 64 deletions ...e/src/main/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStore.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,70 +519,6 @@ private static Map<String, RoleDescriptor> initializeReservedRoles() {
+ "and roles that grant access to Kibana."
)
),
// DEPRECATED: to be removed in 9.0.0
entry(
"data_frame_transforms_admin",
new RoleDescriptor(
"data_frame_transforms_admin",
new String[] { "manage_data_frame_transforms" },
new RoleDescriptor.IndicesPrivileges[] {
RoleDescriptor.IndicesPrivileges.builder()
.indices(
TransformInternalIndexConstants.AUDIT_INDEX_PATTERN,
TransformInternalIndexConstants.AUDIT_INDEX_PATTERN_DEPRECATED,
TransformInternalIndexConstants.AUDIT_INDEX_READ_ALIAS
)
.privileges("view_index_metadata", "read")
.build() },
new RoleDescriptor.ApplicationResourcePrivileges[] {
RoleDescriptor.ApplicationResourcePrivileges.builder()
.application("kibana-*")
.resources("*")
.privileges("reserved_ml_user")
.build() },
null,
null,
MetadataUtils.getDeprecatedReservedMetadata("Please use the [transform_admin] role instead"),
null,
null,
null,
null,
"Grants manage_data_frame_transforms cluster privileges, which enable you to manage transforms. "
+ "This role also includes all Kibana privileges for the machine learning features."
)
),
// DEPRECATED: to be removed in 9.0.0
entry(
"data_frame_transforms_user",
new RoleDescriptor(
"data_frame_transforms_user",
new String[] { "monitor_data_frame_transforms" },
new RoleDescriptor.IndicesPrivileges[] {
RoleDescriptor.IndicesPrivileges.builder()
.indices(
TransformInternalIndexConstants.AUDIT_INDEX_PATTERN,
TransformInternalIndexConstants.AUDIT_INDEX_PATTERN_DEPRECATED,
TransformInternalIndexConstants.AUDIT_INDEX_READ_ALIAS
)
.privileges("view_index_metadata", "read")
.build() },
new RoleDescriptor.ApplicationResourcePrivileges[] {
RoleDescriptor.ApplicationResourcePrivileges.builder()
.application("kibana-*")
.resources("*")
.privileges("reserved_ml_user")
.build() },
null,
null,
MetadataUtils.getDeprecatedReservedMetadata("Please use the [transform_user] role instead"),
null,
null,
null,
null,
"Grants monitor_data_frame_transforms cluster privileges, which enable you to use transforms. "
+ "This role also includes all Kibana privileges for the machine learning features. "
)
),
entry(
"transform_admin",
new RoleDescriptor(
Expand Down
Loading