Skip to content

Commit 18c46b8

Browse files
DOC-1528 Document Schema Registry and ACLs (#448)
* DOC-1528 Document Schema Registry and ACLs * Add section on creds * Add links * Apply suggestions from code review Co-authored-by: Michele Cyran <[email protected]> * Update terraform-provider.adoc --------- Co-authored-by: Michele Cyran <[email protected]>
1 parent 4dd09cc commit 18c46b8

File tree

1 file changed

+191
-9
lines changed

1 file changed

+191
-9
lines changed

modules/manage/pages/terraform-provider.adoc

Lines changed: 191 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,23 +5,27 @@ The https://registry.terraform.io/providers/redpanda-data/redpanda/latest[Redpan
55

66
With the Redpanda Terraform provider, you can manage:
77

8-
* ACLs
9-
* Clusters
10-
* Networks
11-
* Resource groups
12-
* Topics
13-
* Users
8+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/acl[ACLs^]
9+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/cluster[Clusters^]
10+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/network[Networks^]
11+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/resource_group[Resource groups^]
12+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/role_assignments[Role assignments^]
13+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/schema[Schemas^]
14+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/schema_registry_acl[Schema Registry ACLs^]
15+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/serverless_cluster[Serverless clusters^]
16+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/topic[Topics^]
17+
* link:https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/user[Users^]
1418
1519
== Why use Terraform with Redpanda?
1620

1721
* **Simplicity**: Manage all your Redpanda Cloud resources in one place.
1822
* **Automation**: Create and modify resources without manual intervention.
19-
* **Version Control**: Track and roll back changes using version control systems such as GitHub.
23+
* **Version Control**: Track and roll back changes using version control systems, such as GitHub.
2024
* **Scalability**: Scale your infrastructure as your needs grow with minimal effort.
2125

2226
== Understand Terraform configurations
2327

24-
Terraform configurations are written in link:https://developer.hashicorp.com/terraform/language[HCL (HashiCorp Configuration Language)], which is declarative. Here are the main building blocks of a Terraform configuration:
28+
Terraform configurations are written in link:https://developer.hashicorp.com/terraform/language[HCL (HashiCorp Configuration Language)^], which is declarative. Here are the main building blocks of a Terraform configuration:
2529

2630
=== Providers
2731

@@ -422,6 +426,181 @@ resource "redpanda_topic" "example" {
422426
}
423427
----
424428

429+
=== Manage Schema Registry and Schema Registry ACLs
430+
431+
You can also use Terraform to manage data plane resources, such as schemas and access controls, through the Redpanda Schema Registry.
432+
433+
The Redpanda Schema Registry provides centralized management of schemas for producers and consumers, ensuring compatibility and consistency of data serialized with formats such as Avro, Protobuf, or JSON Schema. Using the Redpanda Terraform provider, you can create, update, and delete schemas as well as manage fine-grained access control for Schema Registry resources.
434+
435+
You can use the following Terraform resources:
436+
437+
* `redpanda_schema`: Defines and manages schemas in the Schema Registry.
438+
* `redpanda_schema_registry_acl`: Defines access control policies for Schema Registry subjects or registry-wide operations.
439+
440+
==== Create a schema
441+
442+
The `redpanda_schema` resource registers a schema in the Redpanda Schema Registry. Each schema is associated with a subject, which serves as the logical namespace for schema versioning. When you create or update a schema, Redpanda validates its compatibility level.
443+
444+
[source,hcl]
445+
----
446+
data "redpanda_cluster" "byoc" {
447+
id = "byoc-cluster-id"
448+
}
449+
450+
resource "redpanda_user" "schema_user" {
451+
name = "schema-user"
452+
password = var.schema_password
453+
mechanism = "scram-sha-256"
454+
cluster_api_url = data.redpanda_cluster.byoc.cluster_api_url
455+
allow_deletion = true
456+
}
457+
458+
resource "redpanda_schema" "user_events" {
459+
cluster_id = data.redpanda_cluster.byoc.id
460+
subject = "user_events-value"
461+
schema_type = "AVRO"
462+
463+
schema = jsonencode({
464+
type = "record"
465+
name = "UserEvent"
466+
fields = [
467+
{ name = "user_id", type = "string" },
468+
{ name = "event_type", type = "string" },
469+
{ name = "timestamp", type = "long" }
470+
]
471+
})
472+
473+
username = redpanda_user.schema_user.name
474+
password = var.schema_password
475+
}
476+
----
477+
478+
In this example:
479+
480+
* `cluster_id` identifies the Redpanda cluster where the schema is stored.
481+
* `subject` defines the logical name under which schema versions are registered.
482+
* `schema_type` specifies the serialization type (`AVRO`, `JSON`, or `PROTOBUF`).
483+
* `schema` provides the full schema definition, encoded with `jsonencode()`.
484+
* `username` and `password` authenticate the user to the Schema Registry.
485+
486+
==== Store credentials securely
487+
488+
Store credentials using environment variables or sensitive Terraform variables.
489+
490+
For short-lived credentials or CI/CD usage, use provider-level environment variables:
491+
492+
[source,bash]
493+
----
494+
export REDPANDA_SR_USERNAME=schema-user
495+
export REDPANDA_SR_PASSWORD="your-secret-password"
496+
----
497+
498+
Or, declare a sensitive Terraform variable and inject it at runtime:
499+
500+
[source,hcl]
501+
----
502+
variable "schema_password" {
503+
description = "Password for the Schema Registry user"
504+
sensitive = true
505+
}
506+
----
507+
508+
Then, set the value securely using an environment variable before running Terraform:
509+
510+
[source,bash]
511+
----
512+
export TF_VAR_schema_password="your-secret-password"
513+
----
514+
515+
This avoids committing secrets to source control.
516+
517+
==== Manage Schema Registry ACLs
518+
519+
The `redpanda_schema_registry_acl` resource configures fine-grained access control for Schema Registry subjects or registry-wide operations. Each ACL specifies which principal can perform specific operations on a subject or the registry.
520+
521+
[source,hcl]
522+
----
523+
resource "redpanda_schema_registry_acl" "allow_user_read" {
524+
cluster_id = data.redpanda_cluster.byoc.id
525+
principal = "User:${redpanda_user.schema_user.name}"
526+
resource_type = "SUBJECT" # SUBJECT or REGISTRY
527+
resource_name = "user_events-value"
528+
pattern_type = "LITERAL" # LITERAL or PREFIXED
529+
host = "*"
530+
operation = "READ" # READ, WRITE, DELETE, DESCRIBE, etc.
531+
permission = "ALLOW" # ALLOW or DENY
532+
username = redpanda_user.schema_user.name
533+
password = var.schema_password
534+
}
535+
----
536+
537+
In this example:
538+
539+
* `cluster_id` identifies the cluster that hosts the Schema Registry.
540+
* `principal` specifies the user or service account (for example, `User:alice`).
541+
* `resource_type` determines whether the ACL applies to a specific `SUBJECT` or the entire `REGISTRY`.
542+
* `resource_name` defines the subject name (use `*` for wildcard).
543+
* `pattern_type` controls how the resource name is matched (`LITERAL` or `PREFIXED`).
544+
* `operation` defines the permitted action (`READ`, `WRITE`, `DELETE`, etc.).
545+
* `permission` defines whether the operation is allowed or denied.
546+
* `host` specifies the host filter (typically `"*"` for all hosts).
547+
* `username` and `password` authenticate the principal to the Schema Registry.
548+
549+
TIP: To manage Schema Registry ACLs, the user must have cluster-level `ALTER` permissions. This is typically granted through a Kafka ACL with `ALTER` on the `CLUSTER` resource.
550+
551+
==== Combine schema and ACLs
552+
553+
You can define both the schema and its ACLs in a single configuration to automate schema registration and access setup.
554+
555+
[source,hcl]
556+
----
557+
data "redpanda_cluster" "byoc" {
558+
id = "byoc-cluster-id"
559+
}
560+
561+
resource "redpanda_user" "schema_user" {
562+
name = "schema-user"
563+
password = var.schema_password
564+
mechanism = "scram-sha-256"
565+
cluster_api_url = data.redpanda_cluster.byoc.cluster_api_url
566+
allow_deletion = true
567+
}
568+
569+
resource "redpanda_schema" "user_events" {
570+
cluster_id = data.redpanda_cluster.byoc.id
571+
subject = "user_events-value"
572+
schema_type = "AVRO"
573+
574+
schema = jsonencode({
575+
type = "record"
576+
name = "UserEvent"
577+
fields = [
578+
{ name = "user_id", type = "string" },
579+
{ name = "event_type", type = "string" },
580+
{ name = "timestamp", type = "long" }
581+
]
582+
})
583+
584+
username = redpanda_user.schema_user.name
585+
password = var.schema_password
586+
}
587+
588+
resource "redpanda_schema_registry_acl" "user_events_acl" {
589+
cluster_id = data.redpanda_cluster.byoc.id
590+
principal = "User:${redpanda_user.schema_user.name}"
591+
resource_type = "SUBJECT"
592+
resource_name = redpanda_schema.user_events.subject
593+
pattern_type = "LITERAL"
594+
host = "*"
595+
operation = "READ"
596+
permission = "ALLOW"
597+
username = redpanda_user.schema_user.name
598+
password = var.schema_password
599+
}
600+
----
601+
602+
This configuration registers an Avro schema for the `user_events` subject and grants a service account permission to read it from the Schema Registry.
603+
425604
== Delete resources
426605

427606
Terraform provides a way to clean up your infrastructure when resources are no longer needed. The `terraform destroy` command deletes all the resources defined in your configuration.
@@ -455,4 +634,7 @@ This will delete only the `redpanda_network.example` resource.
455634
== Suggested reading
456635

457636
* https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs[Redpanda Terraform Provider documentation^]
458-
* https://github.com/redpanda-data/terraform-provider-redpanda/tree/main/examples[Redpanda Terraform Provider Examples^]
637+
* https://github.com/redpanda-data/terraform-provider-redpanda/tree/main/examples[Redpanda Terraform Provider examples^]
638+
* https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/schema[Schema resource documentation^]
639+
* https://registry.terraform.io/providers/redpanda-data/redpanda/latest/docs/resources/schema_registry_acl[Schema Registry ACL resource documentation^]
640+

0 commit comments

Comments
 (0)