Skip to content

Commit 841ef0a

Browse files
authored
refactor(s3tables): avoid misleading FileIO::from_path (#1240)
The `FileIO::from_path` only validates a s3 url, and finally calls `FileIOBuilder::new(url.scheme())`. For s3 tables, the table bucket is not a real bucket, and we can't do s3 operations on that. The `FileIO::from_path` here gives a fake impression that the bucket name will be used. Signed-off-by: xxchan <[email protected]> @flaneur2020 @Xuanwo ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Signed-off-by: xxchan <[email protected]>
1 parent c34982a commit 841ef0a

File tree

1 file changed

+2
-14
lines changed

1 file changed

+2
-14
lines changed

crates/catalog/s3tables/src/catalog.rs

Lines changed: 2 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ use aws_sdk_s3tables::operation::get_namespace::GetNamespaceOutput;
2424
use aws_sdk_s3tables::operation::get_table::GetTableOutput;
2525
use aws_sdk_s3tables::operation::list_tables::ListTablesOutput;
2626
use aws_sdk_s3tables::types::OpenTableFormat;
27-
use iceberg::io::FileIO;
27+
use iceberg::io::{FileIO, FileIOBuilder};
2828
use iceberg::spec::{TableMetadata, TableMetadataBuilder};
2929
use iceberg::table::Table;
3030
use iceberg::{
@@ -70,19 +70,7 @@ impl S3TablesCatalog {
7070
let aws_config = create_sdk_config(&config.properties, config.endpoint_url.clone()).await;
7171
let s3tables_client = aws_sdk_s3tables::Client::new(&aws_config);
7272

73-
// parse bucket name from ARN format like: arn:aws:s3:<region>:<account>:bucket/<bucket_name>
74-
let bucket_name = config
75-
.table_bucket_arn
76-
.rsplit(":bucket/")
77-
.next()
78-
.ok_or_else(|| {
79-
Error::new(
80-
ErrorKind::DataInvalid,
81-
format!("Invalid bucket ARN format: {}", config.table_bucket_arn),
82-
)
83-
})?;
84-
85-
let file_io = FileIO::from_path(format!("s3://{}", bucket_name))?
73+
let file_io = FileIOBuilder::new("s3")
8674
.with_props(&config.properties)
8775
.build()?;
8876

0 commit comments

Comments
 (0)