feat: add Internal support for cloud-managed roles (#1230)

* refactor(roles): extract helper functions to reduce duplication

Extract two helper functions in the roles client to eliminate code
duplication and improve maintainability:

- validateRole: Centralizes role validation logic (nil check and empty
  name check) that was duplicated across 5 methods (Has, Create, Delete,
  Update, ClearPrincipals). Includes documentation explaining the
  defensive programming rationale despite Kubernetes name guarantees.

- membersToStringSlice: Converts RoleMember slice to principal string
  slice using "PrincipalType:Name" format. Replaces duplicate conversion
  logic in Update and ClearPrincipals methods, reducing 10+ lines to
  simple function calls.

All existing tests pass, confirming behavior is unchanged.

* refactor(controller): extract isRoleRename helper function

Extract isRoleRename helper function to eliminate duplicated complex
conditional logic in the role controller.

The function encapsulates the three-part condition that determines when
a role rename operation is needed:
- Previous effective name exists (not empty)
- Effective name has changed
- Role is currently managed

This helper is used in two locations:
- SyncResource: Triggers rename workflow when effective name changes
- DeleteResource: Cleans up previous role from incomplete renames

Makes the code more readable and self-documenting by replacing complex
boolean expressions with a clear function name. All tests pass.

* feat(api): add internal role support with prefix handling

Add support for Redpanda internal roles that use the "__" prefix
convention. This allows Kubernetes resources to manage roles that
are internal to Redpanda.

Changes:
- Add Internal boolean field to RoleSpec to mark roles as internal
- Add GetEffectiveRoleName() method that returns "__<name>" when
  Internal is true, otherwise returns "<name>"
- Add EffectiveRoleName field to RoleStatus to track the last
  reconciled role name for detecting renames
- Update GetPrincipal() to use the effective role name
- Add InternalRolePrefix constant ("__")

The effective role name is used throughout the reconciliation logic
to ensure the correct role is created/updated/deleted in Redpanda,
while the status field enables proper cleanup during rename operations.

Generated files updated:
- CRD schema (cluster.redpanda.com_redpandaroles.yaml)
- Apply configurations (rolespec.go, rolestatus.go)
- API documentation (crd-docs.adoc)

* test(roles): add tests for internal role functionality

Add comprehensive test coverage for internal role feature:

API tests (role_types_test.go):
- GetEffectiveRoleName returns name with "__" prefix when Internal=true
- GetEffectiveRoleName returns plain name when Internal=false
- GetPrincipal uses effective role name in principal string

Controller tests (role_controller_test.go):
- TestRoleRename: Validates rename workflow when Internal flag toggles
  - Creates role without internal flag
  - Renames to internal role (adds "__" prefix)
  - Renames back to non-internal role (removes prefix)
  - Verifies old roles are cleaned up after renames
  - Confirms proper role existence at each step

All tests verify that:
- Effective role names are correctly computed
- Role creation uses the effective name
- Renames properly create new and delete old roles
- Status tracking enables cleanup of previous roles

* test(acceptance): add internal role scenarios

Add Gherkin acceptance tests for internal role feature with complete
end-to-end validation.

New scenarios in role-crds.feature:
- "Role with internal flag": Validates that roles with Internal=true
  are created with "__" prefix in Redpanda and can be queried correctly
- "Role rename via internal flag toggle": Validates the rename workflow
  when toggling the Internal flag, ensuring old roles are cleaned up

Test implementation (roles.go):
- Add roleHasInternalFlagSet step to configure Internal field
- Add roleExistsInRedpandaWithName step to verify effective names
- Add roleDoesNotExistInRedpandaWithName step for cleanup validation
- Enhance role existence checking to support both CRD name and
  effective role name validation
- Add internal flag support to role creation helpers

Infrastructure (register.go):
- Register new step definitions for internal role scenarios

These acceptance tests validate the complete workflow including:
- Kubernetes CRD creation with Internal flag
- Redpanda role creation with correct prefix
- Role rename handling and cleanup
- Principal and ACL management with effective names

* chore: add changelog entry for internal role feature

Document the addition of the Internal field to RedpandaRole spec for
managing Redpanda internal roles with "__" prefix.

Author: sago2k8

.changes/unreleased/operator-Added-20260121-180540.yaml

@@ -0,0 +1,4 @@
+project: operator
+kind: Added
+body: Added `Internal` boolean field to RedpandaRole spec to enable managing Redpanda internal roles (prefixed with "__") using standard Kubernetes resource names.
+time: 2026-01-21T18:05:40.867383428+01:00

acceptance/features/role-crds.feature

@@ -284,3 +284,134 @@ Feature: Role CRDs
     And role "swap-role" should not have member "olduser1" in cluster "sasl"
     And role "swap-role" should not have member "olduser2" in cluster "sasl"
     And RedpandaRole "swap-role" should have status field "managedPrincipals" set to "true"
+
+  @skip:gke @skip:aks @skip:eks
+  Scenario: Manage roles with internal flag
+    Given there is no role "__admin-role-k8s" in cluster "sasl"
+    And there are the following pre-existing users in cluster "sasl"
+      | name    | password | mechanism     |
+      | alice   | password | SCRAM-SHA-256 |
+      | bob     | password | SCRAM-SHA-256 |
+    When I apply Kubernetes manifest:
+    """
+    apiVersion: cluster.redpanda.com/v1alpha2
+    kind: RedpandaRole
+    metadata:
+      name: admin-role-k8s
+    spec:
+      cluster:
+        clusterRef:
+          name: sasl
+      internal: true
+      principals:
+        - User:alice
+        - User:bob
+    """
+    And role "admin-role-k8s" is successfully synced
+    Then role "__admin-role-k8s" should exist in cluster "sasl" with effective name "__admin-role-k8s"
+    And role "__admin-role-k8s" should have members "alice and bob" in cluster "sasl" with effective name "__admin-role-k8s"
+    And there should be no role "admin-role-k8s" in cluster "sasl" with effective name "admin-role-k8s"
+
+  @skip:gke @skip:aks @skip:eks
+  Scenario: Manage internal roles with authorization
+    Given there is no role "__reader-role-k8s" in cluster "sasl"
+    And there are the following pre-existing users in cluster "sasl"
+      | name    | password | mechanism     |
+      | charlie | password | SCRAM-SHA-256 |
+    When I create topic "internal-test" in cluster "sasl"
+    And I apply Kubernetes manifest:
+    """
+    apiVersion: cluster.redpanda.com/v1alpha2
+    kind: RedpandaRole
+    metadata:
+      name: reader-role-k8s
+    spec:
+      cluster:
+        clusterRef:
+          name: sasl
+      internal: true
+      principals:
+        - User:charlie
+      authorization:
+        acls:
+          - type: allow
+            resource:
+              type: topic
+              name: internal-
+              patternType: prefixed
+            operations: [Read, Describe]
+    """
+    And role "reader-role-k8s" is successfully synced
+    Then role "__reader-role-k8s" should exist in cluster "sasl" with effective name "__reader-role-k8s"
+    And role "__reader-role-k8s" should have members "charlie" in cluster "sasl" with effective name "__reader-role-k8s"
+    And there should be no role "reader-role-k8s" in cluster "sasl" with effective name "reader-role-k8s"
+    And role "reader-role-k8s" should have ACLs for topic pattern "internal-" in cluster "sasl"
+    And "charlie" should be able to read from topic "internal-test" in cluster "sasl"
+
+  @skip:gke @skip:aks @skip:eks
+  Scenario: Toggle internal flag to rename role with ACLs
+    Given there is no role "rename-test" in cluster "sasl"
+    And there is no role "__rename-test" in cluster "sasl"
+    And there are the following pre-existing users in cluster "sasl"
+      | name    | password | mechanism     |
+      | renameuser | password | SCRAM-SHA-256 |
+    When I create topic "rename-test-topic-before" in cluster "sasl"
+    And I create topic "rename-test-topic-after" in cluster "sasl"
+    And I apply Kubernetes manifest:
+    """
+    apiVersion: cluster.redpanda.com/v1alpha2
+    kind: RedpandaRole
+    metadata:
+      name: rename-test
+    spec:
+      cluster:
+        clusterRef:
+          name: sasl
+      principals:
+        - User:renameuser
+      authorization:
+        acls:
+          - type: allow
+            resource:
+              type: topic
+              name: rename-test-
+              patternType: prefixed
+            operations: [Read, Describe]
+    """
+    And role "rename-test" is successfully synced
+    Then role "rename-test" should exist in cluster "sasl" with effective name "rename-test"
+    And role "rename-test" should have members "renameuser" in cluster "sasl" with effective name "rename-test"
+    And role "rename-test" should have ACLs for topic pattern "rename-test-" in cluster "sasl"
+    And "renameuser" should be able to read from topic "rename-test-topic-before" in cluster "sasl"
+    And RedpandaRole "rename-test" should have status field "effectiveRoleName" set to "rename-test"
+    When I apply Kubernetes manifest:
+    """
+    apiVersion: cluster.redpanda.com/v1alpha2
+    kind: RedpandaRole
+    metadata:
+      name: rename-test
+    spec:
+      cluster:
+        clusterRef:
+          name: sasl
+      internal: true
+      principals:
+        - User:renameuser
+      authorization:
+        acls:
+          - type: allow
+            resource:
+              type: topic
+              name: rename-test-
+              patternType: prefixed
+            operations: [Read, Describe]
+    """
+    And role "rename-test" is successfully synced
+    Then role "__rename-test" should exist in cluster "sasl" with effective name "__rename-test"
+    And role "__rename-test" should have members "renameuser" in cluster "sasl" with effective name "__rename-test"
+    And there should be no role "rename-test" in cluster "sasl" with effective name "rename-test"
+    And role "rename-test" should have ACLs for topic pattern "rename-test-" in cluster "sasl"
+    And "renameuser" should be able to read from topic "rename-test-topic-after" in cluster "sasl"
+    And RedpandaRole "rename-test" should have status field "effectiveRoleName" set to "__rename-test"
+
+

acceptance/steps/register.go

@@ -68,6 +68,11 @@ func init() {
 	framework.RegisterStep(`^RedpandaRole "([^"]*)" should have no members in( vectorized)? cluster "([^"]*)"$`, roleShouldHaveNoMembersInCluster)
 	framework.RegisterStep(`^RedpandaRole "([^"]*)" should have status field "([^"]*)" set to "([^"]*)"$`, redpandaRoleShouldHaveStatusFieldSetTo)
 
+	// Direct testing with effective role names (using the role name as it appears in Redpanda)
+	framework.RegisterStep(`^role "([^"]*)" should exist in( vectorized)? cluster "([^"]*)" with effective name "([^"]*)"$`, roleShouldExistInClusterWithEffectiveName)
+	framework.RegisterStep(`^there should be no role "([^"]*)" in( vectorized)? cluster "([^"]*)" with effective name "([^"]*)"$`, thereShouldBeNoRoleInClusterWithEffectiveName)
+	framework.RegisterStep(`^role "([^"]*)" should have members "([^"]*)" in( vectorized)? cluster "([^"]*)" with effective name "([^"]*)"$`, roleShouldHaveMembersWithEffectiveName)
+
 	// Metrics scenario steps
 	framework.RegisterStep(`^the operator is running$`, operatorIsRunning)
 	framework.RegisterStep(`^its metrics endpoint should reject http request with status code "([^"]*)"$`, requestMetricsEndpointPlainHTTP)