Skip to content

feat: Add table-level compaction commands for R2 Data Catalog #10881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 7, 2025
+359 −43
Merged

feat: Add table-level compaction commands for R2 Data Catalog #10881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .changeset/fine-pans-sit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
"wrangler": minor
---

Add table-level compaction commands for R2 Data Catalog:

- `wrangler r2 bucket catalog compaction enable <bucket> [namespace] [table]`
- `wrangler r2 bucket catalog compaction disable <bucket> [namespace] [table]`

This allows you to enable and disable automatic file compaction for a specific R2 data catalog table.
171 changes: 159 additions & 12 deletions packages/wrangler/src/__tests__/r2.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1312,8 +1312,8 @@ describe("r2", () => {
Control settings for automatic file compaction maintenance jobs for your R2 data catalog [open-beta]

COMMANDS
wrangler r2 bucket catalog compaction enable <bucket> Enable automatic file compaction for your R2 data catalog [open-beta]
wrangler r2 bucket catalog compaction disable <bucket> Disable automatic file compaction for your R2 data catalog [open-beta]
wrangler r2 bucket catalog compaction enable <bucket> [namespace] [table] Enable automatic file compaction for your R2 data catalog or a specific table [open-beta]
wrangler r2 bucket catalog compaction disable <bucket> [namespace] [table] Disable automatic file compaction for your R2 data catalog or a specific table [open-beta]

GLOBAL FLAGS
-c, --config Path to Wrangler configuration file [string]
Expand Down Expand Up @@ -1368,7 +1368,10 @@ describe("r2", () => {
"
⛅️ wrangler x.x.x
──────────────────
✨ Successfully enabled file compaction for the data catalog for bucket 'testBucket'."
✨ Successfully enabled file compaction for the data catalog for bucket 'testBucket'.

Compaction will automatically combine small files into larger ones to improve query performance.
For more details, refer to: https://developers.cloudflare.com/r2/data-catalog/about-compaction/"
`
);
});
Expand All @@ -1381,12 +1384,14 @@ describe("r2", () => {
);
expect(std.out).toMatchInlineSnapshot(`
"
wrangler r2 bucket catalog compaction enable <bucket>
wrangler r2 bucket catalog compaction enable <bucket> [namespace] [table]

Enable automatic file compaction for your R2 data catalog [open-beta]
Enable automatic file compaction for your R2 data catalog or a specific table [open-beta]

POSITIONALS
bucket The name of the bucket which contains the catalog [string] [required]
bucket The name of the bucket which contains the catalog [string] [required]
namespace The namespace containing the table (optional, for table-level compaction) [string]
table The name of the table (optional, for table-level compaction) [string]

GLOBAL FLAGS
-c, --config Path to Wrangler configuration file [string]
Expand All @@ -1398,7 +1403,7 @@ describe("r2", () => {

OPTIONS
--target-size The target size for compacted files in MB (allowed values: 64, 128, 256, 512) [number] [default: 128]
--token A cloudflare api token with access to R2 and R2 Data Catalog which will be used to read/write files for compaction. [string] [required]"
--token A cloudflare api token with access to R2 and R2 Data Catalog (required for catalog-level compaction settings only) [string]"
`);
expect(std.err).toMatchInlineSnapshot(`
"X [ERROR] Not enough non-option arguments: got 0, need at least 1
Expand All @@ -1407,13 +1412,99 @@ describe("r2", () => {
`);
});

it("should error if --token is not provided", async () => {
it("should error if --token is not provided for catalog-level", async () => {
await expect(
runWrangler(
"r2 bucket catalog compaction enable testBucket --target-size 512"
)
).rejects.toThrowErrorMatchingInlineSnapshot(
`[Error: Missing required argument: token]`
`[Error: Token is required for catalog-level compaction. Use --token flag to provide a Cloudflare API token.]`
);
});

it("should enable table compaction without token", async () => {
msw.use(
http.post(
"*/accounts/some-account-id/r2-catalog/testBucket/namespaces/testNamespace/tables/testTable/maintenance-configs",
async ({ request }) => {
const body = await request.json();
expect(request.method).toEqual("POST");
expect(body).toEqual({
compaction: {
state: "enabled",
},
});
return HttpResponse.json(
createFetchResult({ success: true }, true)
);
},
{ once: true }
)
);
await runWrangler(
"r2 bucket catalog compaction enable testBucket testNamespace testTable"
);
expect(std.out).toMatchInlineSnapshot(
`
"
⛅️ wrangler x.x.x
──────────────────
✨ Successfully enabled file compaction for table 'testNamespace.testTable' in bucket 'testBucket'."
`
);
});

it("should enable table compaction with custom target size", async () => {
msw.use(
http.post(
"*/accounts/some-account-id/r2-catalog/testBucket/namespaces/testNamespace/tables/testTable/maintenance-configs",
async ({ request }) => {
const body = await request.json();
expect(request.method).toEqual("POST");
expect(body).toEqual({
compaction: {
state: "enabled",
target_size_mb: 256,
},
});
return HttpResponse.json(
createFetchResult({ success: true }, true)
);
},
{ once: true }
)
);
await runWrangler(
"r2 bucket catalog compaction enable testBucket testNamespace testTable --target-size 256"
);
expect(std.out).toMatchInlineSnapshot(
`
"
⛅️ wrangler x.x.x
──────────────────
✨ Successfully enabled file compaction for table 'testNamespace.testTable' in bucket 'testBucket'."
`
);
});

it("should error if only namespace is provided", async () => {
await expect(
runWrangler(
"r2 bucket catalog compaction enable testBucket testNamespace"
)
).rejects.toThrowErrorMatchingInlineSnapshot(
`[Error: Table name is required when namespace is specified]`
);
});

it("should error if only table is provided", async () => {
// This test ensures that if table is passed as namespace position, it errors properly
await expect(
runWrangler(
'r2 bucket catalog compaction enable testBucket "" testTable'
)
).rejects.toThrowErrorMatchingInlineSnapshot(
`[Error: Namespace is required when table is specified]`
);
});
});
Expand All @@ -1429,12 +1520,14 @@ describe("r2", () => {
);
expect(std.out).toMatchInlineSnapshot(`
"
wrangler r2 bucket catalog compaction disable <bucket>
wrangler r2 bucket catalog compaction disable <bucket> [namespace] [table]

Disable automatic file compaction for your R2 data catalog [open-beta]
Disable automatic file compaction for your R2 data catalog or a specific table [open-beta]

POSITIONALS
bucket The name of the bucket which contains the catalog [string] [required]
bucket The name of the bucket which contains the catalog [string] [required]
namespace The namespace containing the table (optional, for table-level compaction) [string]
table The name of the table (optional, for table-level compaction) [string]

GLOBAL FLAGS
-c, --config Path to Wrangler configuration file [string]
Expand Down Expand Up @@ -1503,6 +1596,60 @@ describe("r2", () => {
Disable cancelled."
`);
});

it("should disable table compaction when confirmed", async () => {
setIsTTY(true);
mockConfirm({
text: "Are you sure you want to disable file compaction for table 'testNamespace.testTable' in bucket 'testBucket'?",
result: true,
});
msw.use(
http.post(
"*/accounts/some-account-id/r2-catalog/testBucket/namespaces/testNamespace/tables/testTable/maintenance-configs",
async ({ request }) => {
const body = await request.json();
expect(request.method).toEqual("POST");
expect(body).toEqual({
compaction: {
state: "disabled",
},
});
return HttpResponse.json(
createFetchResult({ success: true }, true)
);
},
{ once: true }
)
);
await runWrangler(
"r2 bucket catalog compaction disable testBucket testNamespace testTable"
);
expect(std.out).toMatchInlineSnapshot(
`
"
⛅️ wrangler x.x.x
──────────────────
Successfully disabled file compaction for table 'testNamespace.testTable' in bucket 'testBucket'."
`
);
});

it("should cancel table compaction disable when rejected", async () => {
setIsTTY(true);
mockConfirm({
text: "Are you sure you want to disable file compaction for table 'testNamespace.testTable' in bucket 'testBucket'?",
result: false,
});
await runWrangler(
"r2 bucket catalog compaction disable testBucket testNamespace testTable"
);
expect(std.out).toMatchInlineSnapshot(`
"
⛅️ wrangler x.x.x
──────────────────
Disable cancelled."
`);
});
});
});
});
Expand Down
Loading
Loading