Skip to content

feat: replace Redpanda Admin API with Kafka SCRAM API for user management#2325

Merged
weeco merged 1 commit intomasterfrom
ms/user-service-kafka-api
Mar 19, 2026
Merged

feat: replace Redpanda Admin API with Kafka SCRAM API for user management#2325
weeco merged 1 commit intomasterfrom
ms/user-service-kafka-api

Conversation

@weeco
Copy link
Contributor

@weeco weeco commented Mar 18, 2026

Summary

  • Replace Redpanda Admin API with Kafka SCRAM protocol for all user CRUD operations (v1, v1alpha2, REST handlers), removing the hard dependency on the Admin API being configured
  • Unified API: user management now works across Redpanda and non-Redpanda Kafka clusters via DescribeUserSCRAMCredentials / AlterUserSCRAMCredentials Kafka protocol operations
  • Simplifies frontend: no longer need to feature-gate user management behind "Admin API configured" — it's available whenever the Kafka cluster supports SCRAM
  • Proto annotations updated from API_REDPANDA_ADMINAPI_KAFKA to reflect the new API dependency
  • ListUsers now returns the SCRAM mechanism for each user
  • DeleteUser now describes credentials first and deletes all mechanisms, with proper not-found handling

Test plan

  • Unit tests added for v1 user service using kfake (create, list, update, delete)
  • Integration tests updated to use Kafka SCRAM API directly
  • Verify user CRUD works against a Redpanda cluster without Admin API configured
  • Verify user CRUD works against a non-Redpanda Kafka cluster with SCRAM support

@weeco weeco requested review from c-julin and graham-rp March 18, 2026 16:36
@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

The latest Buf updates on your PR. Results from workflow Buf CI / validate (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 19, 2026, 8:35 AM

@weeco weeco force-pushed the ms/user-service-kafka-api branch from e8672e8 to f777361 Compare March 18, 2026 17:06
Copy link
Contributor

@graham-rp graham-rp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to extract some of the duplication from v1 and v1alpha2, but LGTM

return NewConnectErrorFromKafkaErrorCode(kafkaErr.Code, nil)
}
return NewConnectError(
connect.CodeInternal,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is there anything we'd expect that's not an internal error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — the non-*kerr.Error path catches transport/connection-level failures (e.g. broker unreachable, timeouts) which aren't Kafka protocol errors. CodeInternal fits since these are infrastructure issues rather than client mistakes. Added a clarifying comment to the docstring.

// Add pagination
sort.SliceStable(filteredUsers, func(i, j int) bool {
return filteredUsers[i].Name < filteredUsers[j].Name
slices.SortStableFunc(filteredUsers, func(a, b *v1.ListUsersResponse_User) int {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't this already sorted above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right — described.Sorted() already returns users sorted by name, and filtering preserves that order. Removed the redundant sort.

…ment

Replace all Redpanda Admin API calls in the user service (v1, v1alpha2,
and REST handlers) with Kafka SCRAM protocol operations
(DescribeUserSCRAMCredentials / AlterUserSCRAMCredentials). This removes
the dependency on the Redpanda Admin API for user management, making the
user service work across Redpanda and non-Redpanda Kafka clusters.

- Add DescribeUserSCRAMCredentials and AlterUserSCRAMs to console servicer
- Add NewConnectErrorFromKafkaError helper for Kafka error translation
- ListUsers now uses DescribeUserSCRAMCredentials and returns mechanism
- CreateUser/UpdateUser use AlterUserSCRAMs with UpsertSCRAM
- DeleteUser describes credentials first, then deletes all mechanisms
- REST handlers no longer gate behind RedpandaClientProvider
- Remove HasRedpandaAPI from user endpoint compatibility checks
- Update proto annotations from API_REDPANDA_ADMIN to API_KAFKA
- Add unit tests using kfake and update integration tests
@weeco weeco force-pushed the ms/user-service-kafka-api branch from f777361 to 8c2aa69 Compare March 19, 2026 08:35
@weeco weeco merged commit fdb47c7 into master Mar 19, 2026
26 checks passed
@weeco weeco deleted the ms/user-service-kafka-api branch March 19, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants