Support GCS vended credentials in Iceberg REST catalog#28423
Open
kaveti wants to merge 2 commits intotrinodb:masterfrom
Open
Support GCS vended credentials in Iceberg REST catalog#28423kaveti wants to merge 2 commits intotrinodb:masterfrom
kaveti wants to merge 2 commits intotrinodb:masterfrom
Conversation
9ffaa0f to
14c29f7
Compare
findinpath
reviewed
Feb 24, 2026
Contributor
findinpath
left a comment
There was a problem hiding this comment.
Thank you for your contribution - please focus on adding integration tests for the functionality.
lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsFileSystem.java
Show resolved
Hide resolved
.../test/java/io/trino/plugin/iceberg/catalog/rest/TestIcebergRestCatalogFileSystemFactory.java
Show resolved
Hide resolved
findinpath
reviewed
Feb 24, 2026
lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsFileSystem.java
Outdated
Show resolved
Hide resolved
9783451 to
078d848
Compare
Author
|
thank you reviewing the PR. @findinpath i have addressed all your review comments. could you please take a look again. |
e234f58 to
18beb6a
Compare
findinpath
reviewed
Mar 2, 2026
lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsFileSystemFactory.java
Outdated
Show resolved
Hide resolved
findinpath
reviewed
Mar 2, 2026
...erg/src/test/java/io/trino/plugin/iceberg/catalog/rest/TestIcebergGcsVendingRestCatalog.java
Outdated
Show resolved
Hide resolved
findinpath
reviewed
Mar 2, 2026
...erg/src/test/java/io/trino/plugin/iceberg/catalog/rest/TestIcebergGcsVendingRestCatalog.java
Outdated
Show resolved
Hide resolved
findinpath
reviewed
Mar 2, 2026
lib/trino-filesystem-gcs/src/main/java/io/trino/filesystem/gcs/GcsFileSystem.java
Outdated
Show resolved
Hide resolved
Contributor
|
Much needed PR for GCS. Thanks @kaveti. PR LGTM, looks in-line with S3 implementation. |
26201f0 to
d9b6b17
Compare
- Rewrite TestIcebergGcsVendingRestCatalog as TestIcebergGcsVendingRestCatalogConnectorSmokeTest extending BaseIcebergConnectorSmokeTest with real GCS credentials - Create IcebergGcsRestCatalogBackendContainer for GCS-specific REST catalog backend - Rename IcebergRestCatalogBackendContainer to IcebergS3RestCatalogBackendContainer - Remove old test class and GcsCredentialVendingCatalogAdapter - Remove default encryption/decryption key mechanism from GcsFileSystem and GcsFileSystemFactory to align with S3FileSystem and AzureFileSystem (only per-call newEncrypted* methods) - Remove encryption key constants from GcsFileSystemConstants - Remove encryption key mapping from IcebergRestCatalogFileSystemFactory - Add new GCS vending test to cloud-tests profile in pom.xml
d9b6b17 to
d382c1f
Compare
Author
|
@findinpath could you please review again. i have addressed your review comments. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Iceberg REST catalog spec allows the catalog server to return
file IO properties in the load-table response. These properties can
include cloud-specific credentials and configuration that override
the connector's static settings on a per-table basis as "vended credentials."
Trino already supports vended S3 credentials (access key, secret key,
session token), but GCS properties from the REST catalog response are
currently ignored. This means users connecting to a REST catalog that
vends GCS credentials (e.g., Tabular, Polaris, or custom catalog
implementations) cannot use those credentials for GCS-backed tables.
For example, a REST catalog server managing tables across multiple GCP
projects might vend different
gcs.project-idandgcs.oauth2.tokenvalues per table. Without this change, Trino would always use the
statically configured credentials, which may not have access to all
tables.
This change adds support for the following Iceberg GCP properties
from vended credentials:
gcs.oauth2.token/gcs.oauth2.token-expires-at— OAuth2 access tokengcs.project-id— GCP project IDgcs.service.host— custom storage endpointgcs.no-auth— skip authentication (e.g., public buckets)gcs.user-project— requester-pays billing projectgcs.encryption-key/gcs.decryption-key— client-side encryption keysThese properties are extracted from
fileIoPropertiesin[IcebergRestCatalogFileSystemFactory] and passed through
ConnectorIdentityextra credentials to the GCS filesystem layer:no-auth(viaNoCredentials) anduser-project(viasetQuotaProjectId) on theStorageOptionsbuilder.
no-authtakes priority over an OAuth token when bothare present.
passes them as defaults to [GcsFileSystem], which applies them
to [newInputFile] and [newOutputFile] operations. Explicit
encryption keys provided via [newEncryptedInputFile] /
[newEncryptedOutputFile] still take precedence.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:
Fixes #24518