Skip to content

Commit 5f9e351

Browse files
authored
Add iceberg endpoints (#9615)
* Add iceberg endpoints * Add iceberg endpoints * CR Fixes
1 parent c3698ba commit 5f9e351

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

55 files changed

+4560
-0
lines changed

api/swagger.yml

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2038,6 +2038,75 @@ components:
20382038
token:
20392039
type: string
20402040
description: The license JWT token
2041+
2042+
IcebergNamespaceRef:
2043+
description: Reference to one or more levels of a namespace
2044+
type: array
2045+
items:
2046+
type: string
2047+
example: [ "accounting", "tax" ]
2048+
2049+
IcebergRemoteTable:
2050+
type: object
2051+
required:
2052+
- namespace
2053+
- table
2054+
properties:
2055+
namespace:
2056+
$ref: "#/components/schemas/IcebergNamespaceRef"
2057+
table:
2058+
type: string
2059+
description: Remote table name
2060+
2061+
IcebergLocalTable:
2062+
type: object
2063+
required:
2064+
- namespace
2065+
- table
2066+
- repository_id
2067+
- reference_id
2068+
properties:
2069+
namespace:
2070+
$ref: "#/components/schemas/IcebergNamespaceRef"
2071+
table:
2072+
type: string
2073+
description: Remote table name
2074+
repository_id:
2075+
type: string
2076+
description: lakeFS repository ID
2077+
reference_id:
2078+
type: string
2079+
description: lakeFS reference ID (branch or commit)
2080+
2081+
IcebergPushRequest:
2082+
type: object
2083+
required:
2084+
- source
2085+
- destination
2086+
properties:
2087+
source:
2088+
$ref: "#/components/schemas/IcebergLocalTable"
2089+
destination:
2090+
$ref: "#/components/schemas/IcebergRemoteTable"
2091+
force_update:
2092+
type: boolean
2093+
default: false
2094+
description: Override exiting table in remote if exists
2095+
create_namespace:
2096+
type: boolean
2097+
default: false
2098+
description: Creates namespace in remote catalog if not exist
2099+
2100+
IcebergPullRequest:
2101+
type: object
2102+
required:
2103+
- source
2104+
- destination
2105+
properties:
2106+
source:
2107+
$ref: "#/components/schemas/IcebergRemoteTable"
2108+
destination:
2109+
$ref: "#/components/schemas/IcebergLocalTable"
20412110

20422111
paths:
20432112
/setup_comm_prefs:
@@ -6431,3 +6500,57 @@ paths:
64316500
description: failed to exchange authorization code for token
64326501
default:
64336502
$ref: "#/components/responses/ServerError"
6503+
6504+
/iceberg/remotes/{catalog}/push:
6505+
parameters:
6506+
- in: path
6507+
name: catalog
6508+
required: true
6509+
schema:
6510+
type: string
6511+
post:
6512+
tags:
6513+
- remotes
6514+
operationId: pushIcebergTable
6515+
summary: register existing lakeFS table in remote catalog
6516+
requestBody:
6517+
content:
6518+
application/json:
6519+
schema:
6520+
$ref: "#/components/schemas/IcebergPushRequest"
6521+
responses:
6522+
204:
6523+
description: successfully pushed table to remote
6524+
401:
6525+
$ref: "#/components/responses/Unauthorized"
6526+
404:
6527+
$ref: "#/components/responses/NotFound"
6528+
default:
6529+
$ref: "#/components/responses/ServerError"
6530+
6531+
/iceberg/remotes/{catalog}/pull:
6532+
parameters:
6533+
- in: path
6534+
name: catalog
6535+
required: true
6536+
schema:
6537+
type: string
6538+
post:
6539+
tags:
6540+
- remotes
6541+
operationId: pullIcebergTable
6542+
summary: take a table previously pushed from lakeFS into a remote catalog, and pull its state back into the originating lakeFS repository
6543+
requestBody:
6544+
content:
6545+
application/json:
6546+
schema:
6547+
$ref: "#/components/schemas/IcebergPullRequest"
6548+
responses:
6549+
204:
6550+
description: successfully pulled table from remote
6551+
401:
6552+
$ref: "#/components/responses/Unauthorized"
6553+
404:
6554+
$ref: "#/components/responses/NotFound"
6555+
default:
6556+
$ref: "#/components/responses/ServerError"

clients/java/README.md

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

clients/java/api/openapi.yaml

Lines changed: 203 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

clients/java/docs/IcebergLocalTable.md

Lines changed: 16 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)