Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,10 @@
*** xref:manage:rpk/rpk-install.adoc[]
*** xref:manage:rpk/broker-admin.adoc[]
*** xref:manage:rpk/config-rpk-profile.adoc[]
** xref:manage:iceberg/index.adoc[Iceberg]
*** xref:manage:iceberg/about-iceberg-topics.adoc[]
*** xref:manage:iceberg/use-iceberg-catalogs.adoc[]
*** xref:manage:iceberg/query-iceberg-topics.adoc[]
** xref:manage:schema-reg/index.adoc[Schema Registry]
*** xref:manage:schema-reg/schema-reg-overview.adoc[]
*** xref:manage:schema-reg/schema-reg-ui.adoc[]
Expand Down
6 changes: 6 additions & 0 deletions modules/get-started/pages/whats-new-cloud.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ This page lists new features added in Redpanda Cloud.

== April 2025

=== Iceberg topics: beta

The xref:manage:iceberg/about-iceberg-topics.adoc[Iceberg integration for Redpanda] allows you to store topic data in the cloud in the Iceberg open table format. This makes your streaming data immediately available in downstream analytical systems without setting up and maintaining additional ETL pipelines. You can also integrate your data directly into commonly-used big data processing frameworks, standardizing and simplifying the consumption of streams as tables in a wide variety of data analytics pipelines.

Iceberg topics is supported in AWS and GCP.

=== AI agents

Redpanda Cloud is starting to introduce beta versions of xref:develop:agents/about.adoc[AI agents] for enterprise agentic applications driven by a continuous data feed.
Expand Down
7 changes: 7 additions & 0 deletions modules/manage/pages/iceberg/about-iceberg-topics.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
= About Iceberg Topics
:description: Learn how Redpanda can integrate topics with Apache Iceberg.
:page-categories: Iceberg, Integration, Management, High Availability, Data Replication
:page-beta: true
:env-cloud: true

include::25.1@ROOT:manage:partial$iceberg/about-iceberg-topics.adoc[]
4 changes: 4 additions & 0 deletions modules/manage/pages/iceberg/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
= Integrate Redpanda with Iceberg
:description: Generate Iceberg tables for your Redpanda topics for data lakehouse access.
:page-layout: index
:page-beta: true
9 changes: 9 additions & 0 deletions modules/manage/pages/iceberg/query-iceberg-topics.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
= Query Iceberg Topics
:description: Query Redpanda topic data stored in Iceberg tables, based on the topic Iceberg mode and schema.
:page-categories: Iceberg, Integration, Management, High Availability, Data Replication
:page-beta: true
:env-cloud: true

When you access Iceberg topics from a data lakehouse or other Iceberg-compatible tools, how you consume the data depends on the xref:manage:iceberg/about-iceberg-topics.adoc#enable-iceberg-integration[Iceberg mode] you've chosen for a topic, and whether you've registered a schema for the topic in the xref:manage:schema-reg/schema-reg-overview.adoc[Redpanda Schema Registry]. You do not need to rely on complex ETL jobs or pipelines to access real-time data from Redpanda.

include::25.1@ROOT:manage:partial$iceberg/query-iceberg-topics.adoc[]
Copy link
Contributor

@micheleRP micheleRP Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need 25.1 in the directive? Same for all. We should remove if not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the version number caused build errors. I'll keep them here for now.

11 changes: 11 additions & 0 deletions modules/manage/pages/iceberg/use-iceberg-catalogs.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
= Use Iceberg Catalogs
:description: Learn how to access Redpanda topic data stored in Iceberg tables, using table metadata or a catalog integration.
:page-categories: Iceberg, Management, High Availability, Data Replication, Integration
:page-beta: true
:env-cloud: true

include::25.1@ROOT:manage:partial$iceberg/use-iceberg-catalogs.adoc[]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kbatuigas I need to pull in the new 25.1 properties, then they'll get single sourced into cloud. So we might need to go back to update these links later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed in team meeting

== Next steps

* xref:manage:iceberg/query-iceberg-topics.adoc[]
94 changes: 91 additions & 3 deletions modules/manage/partials/controlplane-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ Create a network by making a request to xref:api:ROOT:cloud-controlplane-api.ado

Choose a xref:networking:cidr-ranges.adoc[CIDR range] that does not overlap with your existing VPCs or your Redpanda network.

ifdef::env-dedicated[]
[,bash]
----
curl -d \
Expand All @@ -114,13 +115,29 @@ curl -d \
"region": "us-west1"
}' -H "Authorization: Bearer <token>" -X POST https://api.redpanda.com/v1/networks
----
endif::[]
ifdef::env-byoc[]
[,bash]
----
curl -d \
'{
"cidr_block": "10.0.0.0/20",
"cloud_provider": "CLOUD_PROVIDER_GCP",
"cluster_type": "TYPE_BYOC",
"name": "<network-name>",
"resource_group_id": "<resource-group-id>",
"region": "us-west1"
}' -H "Authorization: Bearer <token>" -X POST https://api.redpanda.com/v1/networks
----
endif::[]

This endpoint returns a <<lro,long-running operation>>.

=== Create a new cluster

After the network is created, make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#post-/v1/clusters[`POST /v1/clusters`] with the resource group ID and network ID in the request body.

ifdef::env-dedicated[]
[,bash]
----
curl -d \
Expand All @@ -137,9 +154,41 @@ curl -d \
"us-west1-a",
"us-west1-b",
"us-west1-c"
]
}' -H "Authorization: Bearer <token>" -X POST https://api.redpanda.com/v1/clusters
],
"cluster_configuration": {
"custom_properties": {
"audit_enabled":true
}
}
}' -H "Authorization: Bearer <token>" -X POST https://api.redpanda.com/v1/clusters
----
endif::[]
ifdef::env-byoc[]
[,bash]
----
curl -d \
'{
"cloud_provider": "CLOUD_PROVIDER_GCP",
"connection_type": "CONNECTION_TYPE_PUBLIC",
"name": "my-new-cluster",
"resource_group_id": "<resource-group-id>",
"network_id": "<network-id>",
"region": "us-west1",
"throughput_tier": "tier-1-gcp-um4g",
"type": "TYPE_BYOC",
"zones": [
"us-west1-a",
"us-west1-b",
"us-west1-c"
],
"cluster_configuration": {
"custom_properties": {
"audit_enabled":true
}
}
}' -H "Authorization: Bearer <token>" -X POST https://api.redpanda.com/v1/clusters
----
endif::[]

The Create Cluster endpoint returns a <<lro,long-running operation>>. When the operation completes, you can retrieve cluster details by calling xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/clusters/-id-[`GET /v1/clusters/\{id}`], and passing the cluster ID as a parameter.

Expand Down Expand Up @@ -264,14 +313,53 @@ The Create Serverless Cluster endpoint returns a <<lro-serverless,long-running o

endif::[]

== Update cluster configuration

To update your cluster configuration properties, make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-id-[`PATCH /v1/clusters/\{id}`] endpoint, passing the cluster ID as a parameter. Include the properties to update in the request body.

ifdef::env-byoc[]
[,bash]
----
curl -H "Authorization: Bearer <token>" \
-H 'accept: application/json'\
-H 'content-type: application/json' \
-d '{
"cluster_configuration": {
"custom_properties": {
"iceberg_enabled":true,
"iceberg_catalog_type":"rest"
}
}
}' -X PATCH "https://api.cloud.redpanda.com/v1/clusters/<cluster-id>"
----
endif::[]

ifndef::env-byoc[]
[,bash]
----
curl -H "Authorization: Bearer <token>" \
-H 'accept: application/json'\
-H 'content-type: application/json' \
-d '{
"cluster_configuration": {
"custom_properties": {
"audit_enabled":true
}
}
}' -X PATCH "https://api.cloud.redpanda.com/v1/clusters/<cluster-id>"
----
endif::[]

The Update Cluster endpoint returns a <<lro,long-running operation>>. <<check-operation-state,Check the operation state>> to verify that the update is complete.

== Delete a cluster

ifndef::env-serverless[]

To delete a cluster, make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#delete-/v1/clusters/-id-[`DELETE /v1/clusters/\{id}`] endpoint, passing the cluster ID as a parameter. This is a <<lro,long-running operation>>.

```bash
curl -H "Authorization: Bearer <token>" -X DELETE https://api.redpanda.com/v1/clusters/<cluster_id>
curl -H "Authorization: Bearer <token>" -X DELETE https://api.redpanda.com/v1/clusters/<cluster-id>
```

ifdef::env-byoc[]
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.