Skip to content

Commit a69d6d8

Browse files
authored
fix: enable s3fs cache_regions for bucket region detection (#582)
## Problem New S3 buckets in different regions fail with `UserAuthError: Access Denied` until AWS DNS/routing converges (2-3 hours) when you try to connect with the wrong region (i.e. use `us-east-1` for a bucket in `us-east-2`). ## Solution Enable `cache_regions=True` in s3fs configuration to handle AWS region redirects automatically. **Before:** Bucket access fails immediately, works after hours of waiting **After:** Bucket access works immediately regardless of region
1 parent 7c6afe0 commit a69d6d8

File tree

3 files changed

+8
-1
lines changed

3 files changed

+8
-1
lines changed

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 1.2.9
2+
3+
**Fix**: enable s3fs cache_regions for bucket region detection
4+
15
## 1.2.8
26

37
**Fix**: Fix artifact url

unstructured_ingest/__version__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "1.2.8" # pragma: no cover
1+
__version__ = "1.2.9" # pragma: no cover

unstructured_ingest/processes/connectors/fsspec/s3.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,9 @@ def get_access_config(self) -> dict[str, Any]:
118118
if self.endpoint_url:
119119
access_configs["endpoint_url"] = self.endpoint_url
120120

121+
# This allows s3fs to properly follow AWS region redirects
122+
access_configs["cache_regions"] = True
123+
121124
return access_configs
122125

123126
@requires_dependencies(["s3fs", "fsspec"], extras="s3")

0 commit comments

Comments
 (0)