Skip to content

Commit 022953b

Browse files
committed
[R2 Data Catalog] Update Docs for Table-Level Compaction Commands
1 parent a4070db commit 022953b

File tree

4 files changed

+94
-15
lines changed

4 files changed

+94
-15
lines changed
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
title: R2 Data Catalog table-level compaction
3+
description: Control compaction settings for individual Iceberg tables
4+
products:
5+
- r2
6+
date: 2025-10-06T13:00:00
7+
hidden: false
8+
---
9+
10+
You can now enable compaction for individual [Apache Iceberg](https://iceberg.apache.org/) tables in [R2 Data Catalog](/r2/data-catalog/), giving you fine-grained control over different workloads.
11+
12+
```bash
13+
# Enable compaction for a specific table (no token required)
14+
npx wrangler r2 bucket catalog compaction enable <BUCKET> <NAMESPACE> <TABLE> --target-size 256
15+
```
16+
17+
This allows you to:
18+
19+
- Apply different target file sizes per table
20+
- Disable compaction for specific tables
21+
- Optimize based on table-specific access patterns
22+
23+
Learn more at [manage catalogs](/r2/data-catalog/manage-catalogs/).

src/content/docs/r2/data-catalog/about-compaction.mdx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ Every write operation in [Apache Iceberg](https://iceberg.apache.org/), no matte
2121

2222
R2 Data Catalog can now [manage compaction](/r2/data-catalog/manage-catalogs) for Apache Iceberg tables stored in R2. When enabled, compaction runs automatically and combines new files that have not been compacted yet.
2323

24+
You can enable compaction at two levels:
25+
- **Catalog-level**: Applies to all tables in your R2 bucket
26+
- **Table-level**: Fine-grained control for specific tables
27+
28+
Table-level settings allow you to customize compaction behavior for different workloads within the same bucket.
29+
2430
Compacted files are prefixed with `compacted-` in the `/data/` directory.
2531

2632
### Choosing the right target file size

src/content/docs/r2/data-catalog/manage-catalogs.mdx

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,10 @@ npx wrangler r2 bucket catalog enable <BUCKET_NAME>
4949
```
5050

5151
After enabling, Wrangler will return your catalog URI and warehouse name.
52+
5253
</TabItem>
5354
</Tabs>
5455

55-
5656
## Disable R2 Data Catalog on a bucket
5757

5858
When you disable the catalog on a bucket, it immediately stops serving requests from the catalog interface. Any Iceberg table references stored in that catalog become inaccessible until you re-enable it.
@@ -74,11 +74,14 @@ To disable the catalog on your bucket, run the [`r2 bucket catalog disable comma
7474
```bash
7575
npx wrangler r2 bucket catalog disable <BUCKET_NAME>
7676
```
77+
7778
</TabItem>
7879
</Tabs>
7980

8081
## Enable compaction
82+
8183
Compaction improves query performance by combining the many small files created during data ingestion into fewer, larger files according to the set `target file size`. For more information about compaction and why it's valuable, refer to [About compaction](/r2/data-catalog/about-compaction/).
84+
8285
<Tabs syncKey='CLIvDash'>
8386
<TabItem label='Dashboard'>
8487

@@ -96,23 +99,37 @@ Compaction improves query performance by combining the many small files created
9699
</TabItem>
97100
<TabItem label='Wrangler CLI'>
98101

99-
To enable the compaction on your catalog, run the [`r2 bucket catalog enable command`](/workers/wrangler/commands/#r2-bucket-catalog-compaction-enable):
102+
To enable the compaction on your catalog, run the [`r2 bucket catalog compaction enable command`](/workers/wrangler/commands/#r2-bucket-catalog-compaction-enable):
100103

101104
```bash
102-
npx wrangler r2 bucket catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>
105+
# Enable catalog-level compaction (all tables)
106+
npx wrangler r2 bucket catalog compaction enable <BUCKET_NAME> --target-size 128 --token <API_TOKEN>
107+
108+
# Enable compaction for a specific table
109+
npx wrangler r2 bucket catalog compaction enable <BUCKET_NAME> <NAMESPACE> <TABLE> --target-size 128
103110
```
111+
112+
:::note[Table-level vs Catalog-level compaction]
113+
114+
- **Catalog-level**: Applies to all tables in the bucket, requires an API token as a service credential.
115+
- **Table-level**: Applies to a specific table only.
116+
117+
:::
118+
104119
</TabItem>
105120
</Tabs>
106121

107122
:::note[API token permission requirements]
108123
Compaction requires a Cloudflare API token with both R2 storage and R2 Data Catalog read/write permissions to act as a service credential. The compaction process uses this token to read files, combine them, and update table metadata.
109124

110125
Refer to [Authenticate your Iceberg engine](#authenticate-your-iceberg-engine) for details on creating a token with the required permissions.
126+
:::
111127

112-
Once enabled, compaction applies retroactively to all existing tables and automatically to newly created tables. During open beta, we currently compact up to 2 GB worth of files once per hour for each table.
128+
Once enabled, compaction applies retroactively to all existing tables (for catalog-level) or the specified table (for table-level). During open beta, we currently compact up to 2 GB worth of files once per hour for each table.
113129

114130
## Disable compaction
115-
Disabling compaction will prevent the process from running for all tables managed by the catalog. You can re-enable it at any time.
131+
132+
Disabling compaction will prevent the process from running for all tables (catalog-level) or a specific table (table-level). You can re-enable it at any time.
116133

117134
<Tabs syncKey='CLIvDash'>
118135
<TabItem label='Dashboard'>
@@ -130,11 +147,16 @@ Disabling compaction will prevent the process from running for all tables manage
130147
</TabItem>
131148
<TabItem label='Wrangler CLI'>
132149

133-
To disable the compaction on your catalog, run the [`r2 bucket catalog disable command`](/workers/wrangler/commands/#r2-bucket-catalog-compaction-disable):
150+
To disable the compaction on your catalog, run the [`r2 bucket catalog compaction disable command`](/workers/wrangler/commands/#r2-bucket-catalog-compaction-disable):
134151

135152
```bash
153+
# Disable catalog-level compaction (all tables)
136154
npx wrangler r2 bucket catalog compaction disable <BUCKET_NAME>
155+
156+
# Disable compaction for a specific table
157+
npx wrangler r2 bucket catalog compaction disable <BUCKET_NAME> <NAMESPACE> <TABLE>
137158
```
159+
138160
</TabItem>
139161
</Tabs>
140162

src/content/partials/workers/wrangler-commands/r2.mdx

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -112,33 +112,61 @@ wrangler r2 bucket catalog get <NAME> [OPTIONS]
112112
depth={3}
113113
/>
114114

115-
Enable compaction on a [R2 Data Catalog](/r2/data-catalog/).
115+
Enable compaction on a [R2 Data Catalog](/r2/data-catalog/) or a specific table.
116116

117117
```txt
118-
wrangler r2 bucket catalog compaction enable <BUCKET> [OPTIONS]
118+
wrangler r2 bucket catalog compaction enable <BUCKET> [NAMESPACE] [TABLE] [OPTIONS]
119119
```
120120

121121
- `BUCKET` <Type text="string" /> <MetaInfo text="required" />
122122
- The name of the bucket to enable R2 Data Catalog compaction for.
123-
- `--token` <Type text="string" /> <MetaInfo text="required" />
124-
- The R2 API token with R2 Data Catalog edit permissions
125-
- `--target-size` <Type text="number" /> <MetaInfo text="required" />
126-
- The target file size (in MB) compaction will attempt to generate. Default: 128.
123+
- `NAMESPACE` <Type text="string" /> <MetaInfo text="optional" />
124+
- The namespace containing the table (for table-level compaction). Must be provided together with TABLE.
125+
- `TABLE` <Type text="string" /> <MetaInfo text="optional" />
126+
- The name of the table (for table-level compaction). Must be provided together with NAMESPACE.
127+
- `--token` <Type text="string" /> <MetaInfo text="optional" />
128+
- The R2 API token with R2 Data Catalog edit permissions. **Required for catalog-level compaction only.**
129+
- `--target-size` <Type text="number" /> <MetaInfo text="optional" />
130+
- The target file size (in MB) compaction will attempt to generate. Default: 128. Allowed values: 64, 128, 256, 512.
131+
132+
Examples:
133+
134+
```bash
135+
# Enable catalog-level compaction (requires token)
136+
npx wrangler r2 bucket catalog compaction enable my-bucket --token <TOKEN>
137+
138+
# Enable table-level compaction
139+
npx wrangler r2 bucket catalog compaction enable my-bucket my-namespace my-table --target-size 256
140+
```
127141

128142
<AnchorHeading
129143
title="`catalog compaction disable`"
130144
slug="r2-bucket-catalog-compaction-disable"
131145
depth={3}
132146
/>
133147

134-
Disable compaction on a [R2 Data Catalog](/r2/data-catalog/).
148+
Disable compaction on a [R2 Data Catalog](/r2/data-catalog/) or a specific table.
135149

136150
```txt
137-
wrangler r2 bucket catalog compaction disable <BUCKET> [OPTIONS]
151+
wrangler r2 bucket catalog compaction disable <BUCKET> [NAMESPACE] [TABLE] [OPTIONS]
138152
```
139153

140154
- `BUCKET` <Type text="string" /> <MetaInfo text="required" />
141-
- The name of the bucket to enable R2 Data Catalog compaction for.
155+
- The name of the bucket to disable R2 Data Catalog compaction for.
156+
- `NAMESPACE` <Type text="string" /> <MetaInfo text="optional" />
157+
- The namespace containing the table (for table-level compaction). Must be provided together with TABLE.
158+
- `TABLE` <Type text="string" /> <MetaInfo text="optional" />
159+
- The name of the table (for table-level compaction). Must be provided together with NAMESPACE.
160+
161+
Examples:
162+
163+
```bash
164+
# Disable catalog-level compaction
165+
npx wrangler r2 bucket catalog compaction disable my-bucket
166+
167+
# Disable table-level compaction
168+
npx wrangler r2 bucket catalog compaction disable my-bucket my-namespace my-table
169+
```
142170

143171
<AnchorHeading title="`cors set`" slug="r2-bucket-cors-set" depth={3} />
144172

0 commit comments

Comments
 (0)