Skip to content
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion local-antora-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ content:
- url: .
branches: HEAD
- url: https://github.com/redpanda-data/documentation
branches: [main, v/*, api, shared, site-search]
branches: [main, '805-single-source-iceberg', api, shared, site-search]
- url: https://github.com/redpanda-data/redpanda-labs
branches: main
start_paths: [docs,'*/docs']
Expand Down
4 changes: 4 additions & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,10 @@
** xref:develop:transactions.adoc[]

* xref:manage:index.adoc[Manage]
** xref:manage:iceberg/index.adoc[Iceberg]
*** xref:manage:iceberg/about-iceberg-topics.adoc[]
*** xref:manage:iceberg/use-iceberg-catalogs.adoc[]
*** xref:manage:iceberg/query-iceberg-topics.adoc[]
** xref:manage:maintenance.adoc[]
** xref:manage:monitor-cloud.adoc[]
** xref:manage:mountable-topics.adoc[]
Expand Down
8 changes: 8 additions & 0 deletions modules/get-started/pages/whats-new-cloud.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@

This page lists new features added in Redpanda Cloud.

== April 2025

=== Iceberg topics: beta

The xref:manage:iceberg/about-iceberg-topics.adoc[Iceberg integration for Redpanda] allows you to store topic data in the cloud in the Iceberg open table format. This makes your streaming data immediately available in downstream analytical systems without setting up and maintaining additional ETL pipelines. You can also integrate your data directly into commonly-used big data processing frameworks, standardizing and simplifying the consumption of streams as tables in a wide variety of data analytics pipelines.

Iceberg topics is supported in AWS, GCP, and Azure.

== March 2025

=== Cloud API: GA
Expand Down
7 changes: 7 additions & 0 deletions modules/manage/pages/iceberg/about-iceberg-topics.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
= About Iceberg Topics
:description: Learn how Redpanda can integrate topics with Apache Iceberg.
:page-categories: Iceberg, Integration, Management, High Availability, Data Replication
:page-beta: true
:env-byoc: true

include::25.1@ROOT:manage:partial$iceberg/about-iceberg-topics.adoc[]
4 changes: 4 additions & 0 deletions modules/manage/pages/iceberg/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
= Integrate Redpanda with Iceberg
:description: Generate Iceberg tables for your Redpanda topics for data lakehouse access.
:page-layout: index
:page-beta: true
9 changes: 9 additions & 0 deletions modules/manage/pages/iceberg/query-iceberg-topics.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
= Query Iceberg Topics
:description: Query Redpanda topic data stored in Iceberg tables, based on the topic Iceberg mode and schema.
:page-categories: Iceberg, Integration, Management, High Availability, Data Replication
:page-beta: true
:env-byoc: true

When you access Iceberg topics from a data lakehouse or other Iceberg-compatible tools, how you consume the data depends on the xref:manage:iceberg/about-iceberg-topics.adoc#enable-iceberg-integration[Iceberg mode] you've chosen for a topic, and whether you've registered a schema for the topic in the xref:manage:schema-reg/schema-reg-overview.adoc[Redpanda Schema Registry]. In either mode, you do not need to rely on complex ETL jobs or pipelines to access real-time data from Redpanda.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kbatuigas "In any mode" not "in either mode" (since we now have 3 modes)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be updating in a separate PR for 25.1


include::25.1@ROOT:manage:partial$iceberg/query-iceberg-topics.adoc[]
Copy link
Contributor

@micheleRP micheleRP Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need 25.1 in the directive? Same for all. We should remove if not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the version number caused build errors. I'll keep them here for now.

11 changes: 11 additions & 0 deletions modules/manage/pages/iceberg/use-iceberg-catalogs.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
= Use Iceberg Catalogs
:description: Learn how to access Redpanda topic data stored in Iceberg tables, using table metadata or a catalog integration.
:page-categories: Iceberg, Management, High Availability, Data Replication, Integration
:page-beta: true
:env-byoc: true

include::25.1@ROOT:manage:partial$iceberg/use-iceberg-catalogs.adoc[]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kbatuigas I need to pull in the new 25.1 properties, then they'll get single sourced into cloud. So we might need to go back to update these links later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed in team meeting

== Next steps

* xref:manage:iceberg/query-iceberg-topics.adoc[]
41 changes: 40 additions & 1 deletion modules/manage/partials/controlplane-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,53 @@ The Create Serverless Cluster endpoint returns a <<lro-serverless,long-running o

endif::[]

== Update cluster configuration

To update your cluster configuration properties, make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-id-[`PATCH /v1/clusters/\{id}`] endpoint, passing the cluster ID as a parameter. Include the properties to update in the request body.

ifdef::env-byoc[]
[,bash]
----
curl -H "Authorization: Bearer <token>" \
-H 'accept: application/json'\
-H 'content-type: application/json' \
-d '{
"cluster_configuration": {
"custom_properties": {
"iceberg_enabled":true,
"iceberg_catalog_type":"rest"
}
}
}' -X PATCH "https://api.cloud.redpanda.com/v1/clusters/<cluster-id>"
----
endif::[]

ifndef::env-byoc[]
[,bash]
----
curl -H "Authorization: Bearer <token>" \
-H 'accept: application/json'\
-H 'content-type: application/json' \
-d '{
"cluster_configuration": {
"custom_properties": {
"audit_enabled":true
}
}
}' -X PATCH "https://api.cloud.redpanda.com/v1/clusters/<cluster-id>"
----
endif::[]

The Update Cluster endpoint returns a <<lro,long-running operation>>. <<check-operation-state,Check the operation state>> to verify that the update is complete.

== Delete a cluster

ifndef::env-serverless[]

To delete a cluster, make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#delete-/v1/clusters/-id-[`DELETE /v1/clusters/\{id}`] endpoint, passing the cluster ID as a parameter. This is a <<lro,long-running operation>>.

```bash
curl -H "Authorization: Bearer <token>" -X DELETE https://api.redpanda.com/v1/clusters/<cluster_id>
curl -H "Authorization: Bearer <token>" -X DELETE https://api.redpanda.com/v1/clusters/<cluster-id>
```

ifdef::env-byoc[]
Expand Down