Skip to content

Commit 0c97907

Browse files
authored
Add minio example (#241)
* minio example * Add to docs
1 parent a8b90a6 commit 0c97907

File tree

11 files changed

+629
-0
lines changed

11 files changed

+629
-0
lines changed

docs/examples/minio.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Minio
2+
3+
[MinIO](https://github.com/minio/minio) is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license. It's often used for testing or self-hosting S3-compatible storage.
4+
5+
We can run minio locally using docker:
6+
7+
```shell
8+
docker run -p 9000:9000 -p 9001:9001 \
9+
quay.io/minio/minio server /data --console-address ":9001"
10+
```
11+
12+
`obstore` isn't able to create a bucket, so we need to do that manually. We can do that through the minio web UI. After running the above docker command, go to <http://localhost:9001>. Then log in with the credentials `minioadmin`, `minioadmin` for username and password. Then click "Create a Bucket" and create a bucket with the name `"test-bucket"`.
13+
14+
Now we can create an `S3Store` to interact with minio:
15+
16+
```py
17+
import obstore as obs
18+
from obstore.store import S3Store
19+
20+
store = S3Store(
21+
"test-bucket",
22+
aws_endpoint="http://localhost:9000",
23+
access_key_id="minioadmin",
24+
secret_access_key="minioadmin",
25+
aws_virtual_hosted_style_request=False,
26+
client_options={"allow_http": True},
27+
)
28+
29+
# Add files
30+
obs.put(store, "a.txt", b"foo")
31+
obs.put(store, "b.txt", b"bar")
32+
obs.put(store, "c/d.txt", b"baz")
33+
34+
# List files
35+
files = obs.list(store).collect()
36+
print(files)
37+
38+
# Download a file
39+
resp = obs.get(store, "a.txt")
40+
print(resp.bytes())
41+
42+
# Delete a file
43+
obs.delete(store, "a.txt")
44+
```
45+
46+
There's a [full example](https://github.com/developmentseed/obstore/tree/main/examples/minio) in the obstore repository.
File renamed without changes.

examples/minio/.python-version

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12

examples/minio/README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Minio example
2+
3+
[MinIO](https://github.com/minio/minio) is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license. It's often used for testing or self-hosting S3-compatible storage.
4+
5+
We can run minio locally using docker:
6+
7+
```shell
8+
docker run -p 9000:9000 -p 9001:9001 \
9+
quay.io/minio/minio server /data --console-address ":9001"
10+
```
11+
12+
`obstore` isn't able to create a bucket, so we need to do that manually. We can do that through the minio web UI. After running the above docker command, go to <http://localhost:9001>.
13+
14+
Then log in with the credentials `minioadmin`, `minioadmin` for username and password.
15+
16+
Then click "Create a Bucket" and create a bucket with the name `"test-bucket"`.
17+
18+
Now, run the Python script:
19+
20+
```
21+
uv run python main.py
22+
```

examples/minio/main.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
import asyncio
2+
3+
import obstore as obs
4+
from obstore.store import S3Store
5+
6+
7+
async def main():
8+
store = S3Store(
9+
"test-bucket",
10+
aws_endpoint="http://localhost:9000",
11+
access_key_id="minioadmin",
12+
secret_access_key="minioadmin",
13+
aws_virtual_hosted_style_request=False,
14+
client_options={"allow_http": True},
15+
)
16+
17+
print("Put file:")
18+
await obs.put_async(store, "a.txt", b"foo")
19+
await obs.put_async(store, "b.txt", b"bar")
20+
await obs.put_async(store, "c/d.txt", b"baz")
21+
22+
print("\nList files:")
23+
files = await obs.list(store).collect_async()
24+
print(files)
25+
26+
print("\nFetch a.txt")
27+
resp = await obs.get_async(store, "a.txt")
28+
print(await resp.bytes_async())
29+
30+
print("\nDelete a.txt")
31+
await obs.delete_async(store, "a.txt")
32+
33+
34+
if __name__ == "__main__":
35+
asyncio.run(main())

examples/minio/pyproject.toml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
[project]
2+
name = "progress-bar"
3+
version = "0.1.0"
4+
description = "Add your description here"
5+
readme = "README.md"
6+
requires-python = ">=3.11"
7+
dependencies = ["obstore"]
8+
9+
# TODO: remove this; use published obstore
10+
[tool.uv.sources]
11+
obstore = { path = "../../obstore" }
12+
13+
[dependency-groups]
14+
dev = ["ipykernel>=6.29.5"]

examples/minio/uv.lock

Lines changed: 510 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)