-
Notifications
You must be signed in to change notification settings - Fork 267
Open
Description
Great work on Cocoindex so far.
I've been trying to integrate it with my Minio S3 storage instance.
#600 introduced support for event notifications for S3-style storage (e.g. Minio).
However, I kept getting 'dispatch errors' when trying to connect.
The issue seems to be the path based addressing style that Minio requires.
In theory, you can set addressing_style in your AWS config profile,
but I could not seem to get that to propagate through to the config in the rust client.
Adding a force_path_style boolean parameter on the AWS S3 config spec appears to fix the problem.
I have a PR ready that I'll submit momentarily.
Before:
After:
data_scope["documents"] = flow_builder.add_source(
cocoindex.sources.AmazonS3(
bucket_name="mybucket",
included_patterns=["*.pdf"],
binary=True,
redis=RedisNotification(
redis_url="redis://localhost:6379", redis_channel="bucketevents",
),
force_path_style=True,
),
refresh_interval=timedelta(seconds=60),
)
Now the flow ingests no problem:
badmonster0
Metadata
Metadata
Assignees
Labels
No labels