Skip to content

Conversation

GCHQDeveloper028
Copy link
Contributor

Which issue does this PR close?

Closes #1133

Rationale for this change

This is a small change to allow passing a session token through to the object store. There is discussion in the issue (and elsewhere) on switching to a different library, doing so would superseded this change. This change will make using temporary credentials much neater/easier until that change is made however.

What changes are included in this PR?

Add optional parameter to AmazonS3 to pass a session_token through to the builder

Are there any user-facing changes?

new optional parameter to the AmazonS3 object store session_token that passes through to the AmazonS3Builder

@timsaucer
Copy link
Member

Since we don't currently have a way to test this in our CI, have you been able to test this change against files stored in S3?

@GCHQDeveloper028
Copy link
Contributor Author

Since we don't currently have a way to test this in our CI, have you been able to test this change against files stored in S3?

Sorry, yes I have tested this with a script roughly like:

object_store = AmazonS3(
  "example-bucket",
  access_key_id="xxx",
  secret_access_key="yyy",
  session_token="zzz"
)

ctx = SessionContext()
ctx.register_object_store("s3://",  object_store)
ctx.read_csv("s3://example-bucket/test.csv").show()

which worked as expected (and failed without session_token being set).

@timsaucer
Copy link
Member

Great, thanks. There's no reason CI should have an issue. I'll merge this after it finishes.

@timsaucer timsaucer changed the title feat: support session token parameter for AmaoznS3 feat: support session token parameter for AmazonS3 Oct 16, 2025
@timsaucer timsaucer merged commit 6012349 into apache:main Oct 16, 2025
16 checks passed
@timsaucer
Copy link
Member

Thank you @GCHQDeveloper028

@cfis
Copy link

cfis commented Oct 16, 2025

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support S3 session_token

3 participants