Skip to content

Mention from_url in top-level getting started docs #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kylebarron merged 1 commit into from
Feb 3, 2025
Merged

Mention from_url in top-level getting started docs #201

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/cookbook.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ obs.put(store2, path2, Path("temporary_file"))
It's easy to **stream** a download from one store directly as the upload to another. Only the given

!!! note
Using the async API is currently required to use streaming copies.
Using the async API is currently required to use streaming copies.
Copy link
Member Author

@kylebarron kylebarron Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vscode auto formatter for markdown doesn't keep this indented 😕

Copy link
Contributor

@daviewales daviewales Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try adding a newline before. ( MkDocs will still format it correctly.)


```py
import obstore as obs
Expand Down Expand Up @@ -274,6 +274,6 @@ await obs.put_async(
This will start up to 12 concurrent uploads, each with around 5MB chunks, giving a total memory usage of up to _roughly_ 60MB for this copy.

!!! note
You may need to increase the download timeout for large source files. The timeout defaults to 30 seconds, which may not be long enough to upload the file to the destination.
You may need to increase the download timeout for large source files. The timeout defaults to 30 seconds, which may not be long enough to upload the file to the destination.

You may set the [`timeout` parameter][obstore.store.ClientConfig] in the `client_options` passed when creating the store.
12 changes: 9 additions & 3 deletions docs/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ There are two parts to `obstore`:

## Constructing a store

Classes to construct a store are exported from the `obstore.store` submodule:
You can use the top-level [`from_url`][obstore.store.from_url] function to construct a store from a top-level URL. Among others this supports `s3://bucket/path`, `gs://bucket/path`, `az://account/container/path`, and `https://mydomain/path`.

Alternatively, you can construct a store directly:

- [`S3Store`][obstore.store.S3Store]: Configure a connection to Amazon S3.
- [`GCSStore`][obstore.store.GCSStore]: Configure a connection to Google Cloud Storage.
Expand All @@ -20,14 +22,18 @@ Each store concept has a variety of constructors, and a host of configuration op

**Example:**

For example, creating an anonymous `S3Store` (without any credentials, for use with fully public buckets):
For example, multiple ways to create an anonymous `S3Store` client (without any credentials, for use with fully public buckets):

```py
from obstore.store import S3Store
from obstore.store import S3Store, from_url

from_url("s3://bucket-name", region="us-east-1", skip_signature=True)
from_url("https://bucket-name.s3.us-east-1.amazonaws.com", skip_signature=True)
store = S3Store("bucket-name", region="us-east-1", skip_signature=True)
```

The process is similar for `GCSStore` and `AzureStore`.

### Configuration

Each store class above has its own store-specific configuration. Elements of the store configuration can be passed as keyword arguments or as a dictionary through the `config` named parameter.
Expand Down