-
Notifications
You must be signed in to change notification settings - Fork 37
Endpoint with custom port requires legacy configuration #129
Copy link
Copy link
Open
Description
When working with some self-hosted S3 providers (such as Garage), it can be useful to set a custom port.
Using what I think is the recommended way of doing things, I can create the default secret and try to use it like so:
create or replace secret (type s3, provider config, key_id 'abc123', secret 'abc123', region 'garage',
endpoint 'somehost:3900', url_style 'path', use_ssl 'false';
select count(*) from read_parquet(['s3://bucket/Example/somefile.parquet']);
IO Error:
Could not establish connection error for HTTP HEAD to 'http://somehost:3900/bucket/Example/somefile.parquet'
LINE 1: select count(*) from read_parquet(['s3://bucket/Example/somefile...
^
Digging deeper, I could see no attempt to connect in my logs and I believe that the port number is not being split from the hostname -- but I'm not sure of that because I'm unfamiliar with the AWS SDK and haven't spent a lot of time reading the code.
I did notice however, that running drop secret __default_s3 (or restarting the DuckDB CLI -- just unconfiguring what we've just done) then configuring instead:
set s3_region = 'garage';
set s3_endpoint = 'somehost:3900';
set s3_use_ssl = false;
set s3_url_style = 'path';
set s3_access_key_id = 'abc123';
set s3_secret_access_key = 'abc123';
Allows the previous count(*) command to work correctly.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels