Skip to content

Commit 4a71abe

Browse files
authored
feat: scaffolding work of rest catalog client (#296)
This PR introduces the foundational scaffolding for the Iceberg REST Catalog client implementation in C++. It establishes the core infrastructure for communicating with Iceberg REST Catalog servers following the [Iceberg REST Catalog Specification](https://iceberg.apache.org/spec/#rest-catalog). ### Key Components Added #### 1. HTTP Client Infrastructure (`http_client.h/cc`) * Implemented `HttpClient` class wrapping the CPR library for HTTP operations. * Supports `GET`, `POST`, `POST` (form-urlencoded), `HEAD`, and `DELETE` methods. * Thread-safe session management with mutex protection. * `HttpResponse` wrapper to abstract underlying HTTP library implementation. #### 2. Configuration Management (`catalog_properties.h/cc`) * `RestCatalogProperties` class for REST catalog configuration. #### 3. Resource Path Construction (`resource_paths.h/cc`) * `ResourcePaths` class for building REST API endpoint URLs. #### 4. Error Handling Framework (`error_handlers.h/cc`) * Hierarchical error handler design following the REST specification. * HTTP status code to ErrorKind mapping. * Also, extended `result.h` with new rest related error kinds. #### 5. REST Utilities (`rest_util.h/cc`) * URL encoding/decoding (RFC 3986 compliant via libcurl). * Namespace encoding/decoding with ASCII Unit Separator (`0x1F`). * Configuration merging with proper precedence (server overrides > client config > server defaults). * String utilities (e.g., `TrimTrailingSlash`). #### 7. RestCatalog Implementation (`rest_catalog.h/cc`) * `RestCatalog` class implementing the Catalog interface. * **Initialization workflow:** 1. Validates client configuration. 2. Fetches server configuration from `/v1/config`. 3. Merges server and client properties. 4. Updates resource paths based on final configuration. ### Testing * `rest_util_test.cc`: Comprehensive tests for URL encoding/decoding, namespace encoding, config merging. * `rest_catalog_test.cc`: These currently introduced tests are merely temporary integration tests that require a local REST server, such as the `apache/iceberg-rest-fixture` Docker image While this enables local testing, it is incompatible with the GitHub CI process, so they have been marked as `DISABLED_`. In the future, we aim to follow the example of iceberg-rust by designing comprehensive integration tests to verify the REST catalog client's behavior and integrating them into our GitHub CI pipeline. This work is scheduled for later completion; please refer to issue #333.
1 parent af746c7 commit 4a71abe

24 files changed

+1921
-116
lines changed

src/iceberg/catalog/rest/CMakeLists.txt

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,14 @@
1515
# specific language governing permissions and limitations
1616
# under the License.
1717

18-
set(ICEBERG_REST_SOURCES rest_catalog.cc json_internal.cc)
18+
set(ICEBERG_REST_SOURCES
19+
rest_catalog.cc
20+
catalog_properties.cc
21+
error_handlers.cc
22+
http_client.cc
23+
json_internal.cc
24+
resource_paths.cc
25+
rest_util.cc)
1926

2027
set(ICEBERG_REST_STATIC_BUILD_INTERFACE_LIBS)
2128
set(ICEBERG_REST_SHARED_BUILD_INTERFACE_LIBS)
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one
3+
* or more contributor license agreements. See the NOTICE file
4+
* distributed with this work for additional information
5+
* regarding copyright ownership. The ASF licenses this file
6+
* to you under the Apache License, Version 2.0 (the
7+
* "License"); you may not use this file except in compliance
8+
* with the License. You may obtain a copy of the License at
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on an
14+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
20+
#include "iceberg/catalog/rest/catalog_properties.h"
21+
22+
#include <string_view>
23+
24+
namespace iceberg::rest {
25+
26+
std::unique_ptr<RestCatalogProperties> RestCatalogProperties::default_properties() {
27+
return std::unique_ptr<RestCatalogProperties>(new RestCatalogProperties());
28+
}
29+
30+
std::unique_ptr<RestCatalogProperties> RestCatalogProperties::FromMap(
31+
const std::unordered_map<std::string, std::string>& properties) {
32+
auto rest_catalog_config =
33+
std::unique_ptr<RestCatalogProperties>(new RestCatalogProperties());
34+
rest_catalog_config->configs_ = properties;
35+
return rest_catalog_config;
36+
}
37+
38+
std::unordered_map<std::string, std::string> RestCatalogProperties::ExtractHeaders()
39+
const {
40+
return Extract(kHeaderPrefix);
41+
}
42+
43+
Result<std::string_view> RestCatalogProperties::Uri() const {
44+
auto it = configs_.find(kUri.key());
45+
if (it == configs_.end() || it->second.empty()) {
46+
return InvalidArgument("Rest catalog configuration property 'uri' is required.");
47+
}
48+
return it->second;
49+
}
50+
51+
} // namespace iceberg::rest
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one
3+
* or more contributor license agreements. See the NOTICE file
4+
* distributed with this work for additional information
5+
* regarding copyright ownership. The ASF licenses this file
6+
* to you under the Apache License, Version 2.0 (the
7+
* "License"); you may not use this file except in compliance
8+
* with the License. You may obtain a copy of the License at
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on an
14+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
20+
#pragma once
21+
22+
#include <memory>
23+
#include <string>
24+
#include <unordered_map>
25+
26+
#include "iceberg/catalog/rest/iceberg_rest_export.h"
27+
#include "iceberg/result.h"
28+
#include "iceberg/util/config.h"
29+
30+
/// \file iceberg/catalog/rest/catalog_properties.h
31+
/// \brief RestCatalogProperties implementation for Iceberg REST API.
32+
33+
namespace iceberg::rest {
34+
35+
/// \brief Configuration class for a REST Catalog.
36+
class ICEBERG_REST_EXPORT RestCatalogProperties
37+
: public ConfigBase<RestCatalogProperties> {
38+
public:
39+
template <typename T>
40+
using Entry = const ConfigBase<RestCatalogProperties>::Entry<T>;
41+
42+
/// \brief The URI of the REST catalog server.
43+
inline static Entry<std::string> kUri{"uri", ""};
44+
/// \brief The name of the catalog.
45+
inline static Entry<std::string> kName{"name", ""};
46+
/// \brief The warehouse path.
47+
inline static Entry<std::string> kWarehouse{"warehouse", ""};
48+
/// \brief The optional prefix for REST API paths.
49+
inline static Entry<std::string> kPrefix{"prefix", ""};
50+
/// \brief The prefix for HTTP headers.
51+
inline static constexpr std::string_view kHeaderPrefix = "header.";
52+
53+
/// \brief Create a default RestCatalogProperties instance.
54+
static std::unique_ptr<RestCatalogProperties> default_properties();
55+
56+
/// \brief Create a RestCatalogProperties instance from a map of key-value pairs.
57+
static std::unique_ptr<RestCatalogProperties> FromMap(
58+
const std::unordered_map<std::string, std::string>& properties);
59+
60+
/// \brief Returns HTTP headers to be added to every request.
61+
std::unordered_map<std::string, std::string> ExtractHeaders() const;
62+
63+
/// \brief Get the URI of the REST catalog server.
64+
/// \return The URI if configured, or an error if not set or empty.
65+
Result<std::string_view> Uri() const;
66+
67+
private:
68+
RestCatalogProperties() = default;
69+
};
70+
71+
} // namespace iceberg::rest
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one
3+
* or more contributor license agreements. See the NOTICE file
4+
* distributed with this work for additional information
5+
* regarding copyright ownership. The ASF licenses this file
6+
* to you under the Apache License, Version 2.0 (the
7+
* "License"); you may not use this file except in compliance
8+
* with the License. You may obtain a copy of the License at
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on an
14+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
20+
#pragma once
21+
22+
#include <string>
23+
24+
#include "iceberg/version.h"
25+
26+
/// \file iceberg/catalog/rest/constant.h
27+
/// Constant values for Iceberg REST API.
28+
29+
namespace iceberg::rest {
30+
31+
inline const std::string kHeaderContentType = "Content-Type";
32+
inline const std::string kHeaderAccept = "Accept";
33+
inline const std::string kHeaderXClientVersion = "X-Client-Version";
34+
inline const std::string kHeaderUserAgent = "User-Agent";
35+
36+
inline const std::string kMimeTypeApplicationJson = "application/json";
37+
inline const std::string kMimeTypeFormUrlEncoded = "application/x-www-form-urlencoded";
38+
inline const std::string kUserAgentPrefix = "iceberg-cpp/";
39+
inline const std::string kUserAgent = "iceberg-cpp/" ICEBERG_VERSION_STRING;
40+
41+
inline const std::string kQueryParamParent = "parent";
42+
inline const std::string kQueryParamPageToken = "page_token";
43+
44+
} // namespace iceberg::rest
Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one
3+
* or more contributor license agreements. See the NOTICE file
4+
* distributed with this work for additional information
5+
* regarding copyright ownership. The ASF licenses this file
6+
* to you under the Apache License, Version 2.0 (the
7+
* "License"); you may not use this file except in compliance
8+
* with the License. You may obtain a copy of the License at
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on an
14+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
20+
#include "iceberg/catalog/rest/error_handlers.h"
21+
22+
#include <string_view>
23+
24+
#include "iceberg/catalog/rest/types.h"
25+
26+
namespace iceberg::rest {
27+
28+
namespace {
29+
30+
constexpr std::string_view kIllegalArgumentException = "IllegalArgumentException";
31+
constexpr std::string_view kNoSuchNamespaceException = "NoSuchNamespaceException";
32+
constexpr std::string_view kNamespaceNotEmptyException = "NamespaceNotEmptyException";
33+
34+
} // namespace
35+
36+
const std::shared_ptr<DefaultErrorHandler>& DefaultErrorHandler::Instance() {
37+
static const std::shared_ptr<DefaultErrorHandler> instance{new DefaultErrorHandler()};
38+
return instance;
39+
}
40+
41+
Status DefaultErrorHandler::Accept(const ErrorModel& error) const {
42+
switch (error.code) {
43+
case 400:
44+
if (error.type == kIllegalArgumentException) {
45+
return InvalidArgument(error.message);
46+
}
47+
return BadRequest("Malformed request: {}", error.message);
48+
case 401:
49+
return NotAuthorized("Not authorized: {}", error.message);
50+
case 403:
51+
return Forbidden("Forbidden: {}", error.message);
52+
case 405:
53+
case 406:
54+
break;
55+
case 500:
56+
return InternalServerError("Server error: {}: {}", error.type, error.message);
57+
case 501:
58+
return NotSupported(error.message);
59+
case 503:
60+
return ServiceUnavailable("Service unavailable: {}", error.message);
61+
}
62+
63+
return RestError("Code: {}, message: {}", error.code, error.message);
64+
}
65+
66+
const std::shared_ptr<NamespaceErrorHandler>& NamespaceErrorHandler::Instance() {
67+
static const std::shared_ptr<NamespaceErrorHandler> instance{
68+
new NamespaceErrorHandler()};
69+
return instance;
70+
}
71+
72+
Status NamespaceErrorHandler::Accept(const ErrorModel& error) const {
73+
switch (error.code) {
74+
case 400:
75+
if (error.type == kNamespaceNotEmptyException) {
76+
return NamespaceNotEmpty(error.message);
77+
}
78+
return BadRequest("Malformed request: {}", error.message);
79+
case 404:
80+
return NoSuchNamespace(error.message);
81+
case 409:
82+
return AlreadyExists(error.message);
83+
case 422:
84+
return RestError("Unable to process: {}", error.message);
85+
}
86+
87+
return DefaultErrorHandler::Accept(error);
88+
}
89+
90+
const std::shared_ptr<DropNamespaceErrorHandler>& DropNamespaceErrorHandler::Instance() {
91+
static const std::shared_ptr<DropNamespaceErrorHandler> instance{
92+
new DropNamespaceErrorHandler()};
93+
return instance;
94+
}
95+
96+
Status DropNamespaceErrorHandler::Accept(const ErrorModel& error) const {
97+
if (error.code == 409) {
98+
return NamespaceNotEmpty(error.message);
99+
}
100+
101+
return NamespaceErrorHandler::Accept(error);
102+
}
103+
104+
const std::shared_ptr<TableErrorHandler>& TableErrorHandler::Instance() {
105+
static const std::shared_ptr<TableErrorHandler> instance{new TableErrorHandler()};
106+
return instance;
107+
}
108+
109+
Status TableErrorHandler::Accept(const ErrorModel& error) const {
110+
switch (error.code) {
111+
case 404:
112+
if (error.type == kNoSuchNamespaceException) {
113+
return NoSuchNamespace(error.message);
114+
}
115+
return NoSuchTable(error.message);
116+
case 409:
117+
return AlreadyExists(error.message);
118+
}
119+
120+
return DefaultErrorHandler::Accept(error);
121+
}
122+
123+
const std::shared_ptr<ViewErrorHandler>& ViewErrorHandler::Instance() {
124+
static const std::shared_ptr<ViewErrorHandler> instance{new ViewErrorHandler()};
125+
return instance;
126+
}
127+
128+
Status ViewErrorHandler::Accept(const ErrorModel& error) const {
129+
switch (error.code) {
130+
case 404:
131+
if (error.type == kNoSuchNamespaceException) {
132+
return NoSuchNamespace(error.message);
133+
}
134+
return NoSuchView(error.message);
135+
case 409:
136+
return AlreadyExists(error.message);
137+
}
138+
139+
return DefaultErrorHandler::Accept(error);
140+
}
141+
142+
const std::shared_ptr<TableCommitErrorHandler>& TableCommitErrorHandler::Instance() {
143+
static const std::shared_ptr<TableCommitErrorHandler> instance{
144+
new TableCommitErrorHandler()};
145+
return instance;
146+
}
147+
148+
Status TableCommitErrorHandler::Accept(const ErrorModel& error) const {
149+
switch (error.code) {
150+
case 404:
151+
return NoSuchTable(error.message);
152+
case 409:
153+
return CommitFailed("Commit failed: {}", error.message);
154+
case 500:
155+
case 502:
156+
case 503:
157+
case 504:
158+
return CommitStateUnknown("Service failed: {}: {}", error.code, error.message);
159+
}
160+
161+
return DefaultErrorHandler::Accept(error);
162+
}
163+
164+
const std::shared_ptr<ViewCommitErrorHandler>& ViewCommitErrorHandler::Instance() {
165+
static const std::shared_ptr<ViewCommitErrorHandler> instance{
166+
new ViewCommitErrorHandler()};
167+
return instance;
168+
}
169+
170+
Status ViewCommitErrorHandler::Accept(const ErrorModel& error) const {
171+
switch (error.code) {
172+
case 404:
173+
return NoSuchView(error.message);
174+
case 409:
175+
return CommitFailed("Commit failed: {}", error.message);
176+
case 500:
177+
case 502:
178+
case 503:
179+
case 504:
180+
return CommitStateUnknown("Service failed: {}: {}", error.code, error.message);
181+
}
182+
183+
return DefaultErrorHandler::Accept(error);
184+
}
185+
186+
} // namespace iceberg::rest

0 commit comments

Comments
 (0)