CORE-14880: Migrate "OIDC" Admin API v1 Security Endpoints to v2 ConnectRPC API #28973

nguyen-andrew · 2025-12-13T07:00:22Z

This PR introduces the security service for Redpanda Admin API v2. Starting with the OIDC-related RPCs, this v2 service initially aims to achieve functional parity with the v1 API by providing the following:

OIDC Identity Resolution - Resolves OIDC token identity and returns principal mapping with token expiration
Cluster-wide OIDC Key Refresh - Refreshes OIDC keys across all brokers in the cluster
Cluster-wide OIDC Session Revocation - Refreshes OIDC keys and revokes all active OIDC sessions cluster-wide

The implementation has the following characteristics:

For OIDC key refresh and ODIC session revocation, any broker can receive requests and will broadcast the operation across all nodes in the cluster.
For broadcasted/proxied operations, any error produced by a proxied request is returned to the client
Session revocation combines key refresh with active session termination

Backports Required

Release Notes

Features

Added OIDC security service to Admin API v2 with identity resolution, cluster-wide key refresh, and session revocation capabilities. Administrators can now manage OIDC authentication security through the new v2 endpoints with proper cluster coordination and error handling.

Copilot

Pull request overview

This PR migrates OIDC-related Admin API endpoints from v1 to v2 using ConnectRPC, introducing the new security service for Admin API v2. The implementation provides functional parity with v1 endpoints while adding proper cluster-wide coordination and error aggregation.

Key changes:

Introduces v2 SecurityService with three OIDC RPCs: identity resolution, cluster-wide key refresh, and session revocation
Implements cluster-wide broadcasting for key refresh and session revocation operations with aggregated error handling
Adds comprehensive integration tests validating v2 endpoints alongside existing v1 tests

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/rptest/tests/redpanda_oauth_test.py	Adds v2 endpoint tests (resolve identity, refresh keys, revoke sessions) and refactors metrics helpers for reuse
src/v/redpanda/application.cc	Registers the new security service in the admin server
src/v/redpanda/admin/services/security.h	Defines security service interface with OIDC RPC methods
src/v/redpanda/admin/services/security.cc	Implements security service with cluster-wide operation broadcasting and error aggregation
src/v/redpanda/admin/services/BUILD	Adds build target for security service library
src/v/redpanda/BUILD	Adds security service dependency to redpanda application
proto/redpanda/core/admin/v2/security.proto	Defines protobuf schema for security service RPCs
proto/redpanda/core/admin/v2/BUILD	Adds build target for security proto library

tests/rptest/tests/redpanda_oauth_test.py

src/v/redpanda/admin/services/security.cc

vbotbuildovich · 2025-12-13T09:11:29Z

CI test results

test results on build#77822

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
PartitionReassignmentsTest	test_reassignments_cancel	null	integration	https://buildkite.com/redpanda/redpanda/builds/77822#019b16a8-20c8-4f42-a2cc-13b1ae5a0406	FLAKY	9/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.1167, p0=0.7110, reject_threshold=0.0100. adj_baseline=0.3109, p1=0.1331, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionReassignmentsTest&test_method=test_reassignments_cancel

test results on build#77928

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
CrashLoopChecksTest	test_cloud_storage_client_misconfiguration	null	integration	https://buildkite.com/redpanda/redpanda/builds/77928#019b25c2-c795-4100-8e31-f6edd8f17288	FAIL	0/51	Test PASSES after retries.Inconclusive result after max retries(baseline=0.9539, p0=0.0944, reject_threshold=0.0100. adj_baseline=0.9999, p1=1.0000, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=CrashLoopChecksTest&test_method=test_cloud_storage_client_misconfiguration
CrashLoopChecksTest	test_cloud_storage_client_misconfiguration	null	integration	https://buildkite.com/redpanda/redpanda/builds/77928#019b25c4-18b8-4b9f-a403-6f671ba7ace9	FLAKY	2/21	Test PASSES after retries.No significant increase in flaky rate(baseline=0.9375, p0=0.8741, reject_threshold=0.0100. adj_baseline=0.9998, p1=0.0000, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=CrashLoopChecksTest&test_method=test_cloud_storage_client_misconfiguration
DataMigrationsApiTest	test_higher_level_migration_api	null	integration	https://buildkite.com/redpanda/redpanda/builds/77928#019b25c4-18bc-49a9-91cc-ee03a81f05e9	FLAKY	20/21	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0159, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.0471, p1=0.3812, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DataMigrationsApiTest&test_method=test_higher_level_migration_api
PartitionReassignmentsTest	test_reassignments_cancel	null	integration	https://buildkite.com/redpanda/redpanda/builds/77928#019b25c2-c79c-4ff1-9e2c-076d45e994a5	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0817, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.2256, p1=0.0775, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionReassignmentsTest&test_method=test_reassignments_cancel

test results on build#77977

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
WriteCachingFailureInjectionTest	test_unavoidable_data_loss	null	integration	https://buildkite.com/redpanda/redpanda/builds/77977#019b282f-cfb4-4c82-b053-60c28e559bca	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0580, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1642, p1=0.1664, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionTest&test_method=test_unavoidable_data_loss

src/v/redpanda/admin/services/security.cc

proto/redpanda/core/admin/v2/security.proto

tests/rptest/tests/redpanda_oauth_test.py

src/v/redpanda/admin/services/security.cc

michael-redpanda · 2025-12-15T20:30:43Z

src/v/redpanda/admin/services/security.cc

+    security::oidc::authenticator auth{_controller->get_oidc_service().local()};
+    auto res = auth.authenticate(
+      auth_result->get_password().substr(authz_bearer_prefix.length()));


nit: This seems redundant to re-authenticate the token (which shoudl have already been authenticated) to just get the expiration timeout. I wonder if the you could either expand the auth_result to also contain that information or maybe expose something in security that can do a simple parse of the provided token.

Definitely agree, maybe we can expand auth_result to include an auth_payload or something and use that instead instead of having to decode the token again - I'd like to explore that in a follow-on PR if that's okay.

tests/rptest/tests/redpanda_oauth_test.py

nguyen-andrew · 2025-12-16T05:23:15Z

Force push to address PR comments:

Added documentation to the security.proto
Updated resolve_oidc_identity to use failed_precondition_exception as suggested.
Simplified implementations of refresh_oidc_keys & revoke_oidc_sessions.
Updated the Admin V2 tests added in redpanda_oauth_test.py to use the generated client.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

tests/rptest/clients/admin/proto/redpanda/core/admin/v2/security_pb2_connect.py

proto/redpanda/core/admin/v2/security.proto

src/v/redpanda/admin/services/security.cc

tests/rptest/tests/redpanda_oauth_test.py

vbotbuildovich · 2025-12-16T07:21:13Z

Retry command for Build#77928

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/crash_loop_checks_test.py::CrashLoopChecksTest.test_cloud_storage_client_misconfiguration

nguyen-andrew · 2025-12-16T16:36:28Z

Force push to rebase on latest dev.

nguyen-andrew · 2025-12-16T16:44:42Z

Force push - needed to re-generate ducktape protos with updated proto documentation.

michael-redpanda

looks really nice!

src/v/redpanda/admin/services/security.cc

tests/rptest/tests/redpanda_oauth_test.py

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Copilot · 2025-12-16T19:33:54Z

tests/rptest/clients/admin/proto/redpanda/core/admin/v2/__init__.pyi

 from . import broker_pb2
 from . import cluster_pb2
-from . import internal
 from . import kafka_connections_pb2
+from . import security_pb2
 from . import shadow_link_pb2


The internal import was removed. If this module exists and is used elsewhere in the codebase, this removal could break imports. However, since this appears to be part of a refactoring to add the security module and the file is a .pyi stub file, this may be intentional cleanup of an unused import.

tests/rptest/tests/redpanda_oauth_test.py

rockwotj

Love how this has been simplified since the first pass. Just one additional nit on top of Mike's feedback.

proto/redpanda/core/admin/v2/security.proto

Creating a new proto file for security-related messages in Admin API v2. This new proto initially includes definitions for OIDC-related services and messages.

This commit adds the skeleton implementation of the OIDC security service for the Redpanda Admin API v2. The service includes unimplemented stubs for the following methods: - resolve_oidc_identity - refresh_oidc_keys - revoke_oidc_sessions

nguyen-andrew · 2025-12-16T22:17:24Z

Force push:

Addressing PR comments
Rebase on dev to update to latest changes to application.cc.

Generate Ducktape protobuf clients to test the new Admin API v2 OIDC security endpoints.

nguyen-andrew · 2025-12-16T22:21:48Z

Force push: accidentally forgot to include changes to tests/rptest/clients/admin/proto/redpanda/core/common/v1/tls_pb2.py that the regenerate_ducktape_protos.sh script generated.

Add OIDC identity resolution to Admin API v2, updates application wiring, and extends oauth tests to validate the new functionality.

Implement refresh_oidc_keys RPC to refresh OIDC keys by calling any broker. The receiving broker executes the refresh on all local shards and proxies/broadcasts the request to all other nodes. Brokers receiving proxied calls only execute locally and do not rebroadcast to other nodes. The originating node aggregates failures across all nodes/shards and returns an aggregated error if any occurred, otherwise success. Add tests ensuring a single-node call refreshes OIDC keys across the cluster.

Implement RevokeOidcSessions RPC to refresh OIDC keys and revoke all active OIDC sessions across the cluster. The receiving broker refreshes OIDC keys on all local shards, revokes OIDC SASL sessions with kafka::server, and broadcasts the request to all other nodes in the cluster, with each broker performing local key refresh and session revocation. The originating node aggregates errors across all brokers and returns an aggregated error if any node fails, otherwise success. Add comprehensive test coverage validating session revocation with various authentication scenarios. This enables administrators to immediately invalidate all OIDC sessions when security policies change or credentials are compromised.

nguyen-andrew · 2025-12-16T22:36:25Z

Force push to use datetime.timezone.utc instead of datetime.UTC (python type checker seems run a version that doesn't have datetime.UTC).

rockwotj

LGTM, will leave the final approval to Mike

michael-redpanda

lgtm!

Copilot AI review requested due to automatic review settings December 13, 2025 07:00

nguyen-andrew requested review from a team, michael-redpanda and rockwotj as code owners December 13, 2025 07:00

github-actions bot added area/build area/redpanda labels Dec 13, 2025

Copilot AI reviewed Dec 13, 2025

View reviewed changes

nguyen-andrew marked this pull request as draft December 15, 2025 16:23

rockwotj reviewed Dec 15, 2025

View reviewed changes

nguyen-andrew commented Dec 15, 2025

View reviewed changes

tests/rptest/tests/redpanda_oauth_test.py Show resolved Hide resolved

michael-redpanda reviewed Dec 15, 2025

View reviewed changes

nguyen-andrew force-pushed the admin-api-v2-oidc branch from 4e7fbc2 to 46c6bfc Compare December 16, 2025 05:15

nguyen-andrew marked this pull request as ready for review December 16, 2025 05:23

nguyen-andrew requested review from Copilot, michael-redpanda and rockwotj December 16, 2025 05:29

nguyen-andrew self-assigned this Dec 16, 2025

Copilot AI reviewed Dec 16, 2025

View reviewed changes

nguyen-andrew force-pushed the admin-api-v2-oidc branch from 46c6bfc to 2577d2f Compare December 16, 2025 16:34

nguyen-andrew force-pushed the admin-api-v2-oidc branch from 2577d2f to b7104cf Compare December 16, 2025 16:43

nguyen-andrew requested review from graham-rp and sago2k8 December 16, 2025 18:59

michael-redpanda reviewed Dec 16, 2025

View reviewed changes

src/v/redpanda/admin/services/security.cc Outdated Show resolved Hide resolved

src/v/redpanda/admin/services/security.cc Outdated Show resolved Hide resolved

tests/rptest/tests/redpanda_oauth_test.py Show resolved Hide resolved

michael-redpanda requested a review from Copilot December 16, 2025 19:33

Copilot AI reviewed Dec 16, 2025

View reviewed changes

rockwotj reviewed Dec 16, 2025

View reviewed changes

proto/redpanda/core/admin/v2/security.proto Outdated Show resolved Hide resolved

nguyen-andrew force-pushed the admin-api-v2-oidc branch from b7104cf to 3130548 Compare December 16, 2025 21:41

nguyen-andrew added 2 commits December 16, 2025 21:45

proto: Adding security.proto for Admin API v2

a13d7af

Creating a new proto file for security-related messages in Admin API v2. This new proto initially includes definitions for OIDC-related services and messages.

admin: Implement OIDC security service stubs

07f80e0

This commit adds the skeleton implementation of the OIDC security service for the Redpanda Admin API v2. The service includes unimplemented stubs for the following methods: - resolve_oidc_identity - refresh_oidc_keys - revoke_oidc_sessions

nguyen-andrew force-pushed the admin-api-v2-oidc branch from 3130548 to 1024d21 Compare December 16, 2025 22:12

dt/pb: Generate Ducktape protobuf clients

8ea9613

Generate Ducktape protobuf clients to test the new Admin API v2 OIDC security endpoints.

nguyen-andrew force-pushed the admin-api-v2-oidc branch from 1024d21 to f8d54f5 Compare December 16, 2025 22:20

nguyen-andrew added 3 commits December 16, 2025 22:30

admin: OIDC identity resolution in Admin API v2

d3e4c8e

Add OIDC identity resolution to Admin API v2, updates application wiring, and extends oauth tests to validate the new functionality.

nguyen-andrew force-pushed the admin-api-v2-oidc branch from f8d54f5 to 17b7d07 Compare December 16, 2025 22:31

nguyen-andrew requested review from michael-redpanda and rockwotj December 17, 2025 01:07

rockwotj reviewed Dec 17, 2025

View reviewed changes

michael-redpanda approved these changes Dec 17, 2025

View reviewed changes

nguyen-andrew merged commit 4a94f35 into redpanda-data:dev Dec 17, 2025
30 checks passed

nguyen-andrew deleted the admin-api-v2-oidc branch December 17, 2025 20:39

CORE-14880: Migrate "OIDC" Admin API v1 Security Endpoints to v2 ConnectRPC API #28973

CORE-14880: Migrate "OIDC" Admin API v1 Security Endpoints to v2 ConnectRPC API #28973

Uh oh!

Conversation

nguyen-andrew commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backports Required

Release Notes

Features

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vbotbuildovich commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI test results

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michael-redpanda Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

nguyen-andrew Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nguyen-andrew commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vbotbuildovich commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Retry command for Build#77928

Uh oh!

nguyen-andrew commented Dec 16, 2025

Uh oh!

nguyen-andrew commented Dec 16, 2025

Uh oh!

michael-redpanda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rockwotj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nguyen-andrew commented Dec 16, 2025

Uh oh!

nguyen-andrew commented Dec 13, 2025 •

edited

Loading

vbotbuildovich commented Dec 13, 2025 •

edited

Loading

nguyen-andrew commented Dec 16, 2025 •

edited

Loading

vbotbuildovich commented Dec 16, 2025 •

edited

Loading