-
Notifications
You must be signed in to change notification settings - Fork 133
chore: update general resource access docs #1988
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 13 commits
57e2bbb
e90ba12
540ca2d
b57d05a
8e7188d
f72ff28
aaf5273
992f083
57f3c96
fe6b89e
30801e5
b89e673
af5d0ff
49d39ee
6057eed
0938857
e609827
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -47,7 +47,9 @@ The default setting strikes a good balance for casual or internal use, but **Res | |||||||||||||||
| You can switch to **Restricted** access at any time. If it causes issues in your workflow, you can revert to the default setting just as easily. | ||||||||||||||||
|
|
||||||||||||||||
| :::note Support in public Actors | ||||||||||||||||
|
|
||||||||||||||||
| Because this is a new setting, some existing public Actors and integrations might not support it yet. Their authors need to update them to provide a valid token on all API calls. | ||||||||||||||||
|
|
||||||||||||||||
| ::: | ||||||||||||||||
|
|
||||||||||||||||
|
|
||||||||||||||||
|
|
@@ -125,25 +127,122 @@ await datasetClient.update({ | |||||||||||||||
|
|
||||||||||||||||
| Even when a resource is restricted, you might still want to share it with someone outside your team — for example, to send a PDF report to a client, or include a screenshot in an automated email or Slack message. In these cases, **storage resources** (like key-value stores, datasets, and request queues) support generating **pre-signed URLs**. These are secure, time-limited links that let others access individual files without needing an Apify account or authentication. | ||||||||||||||||
|
|
||||||||||||||||
| Pre-signed URLs: | ||||||||||||||||
| #### How pre-signed URLs work | ||||||||||||||||
|
|
||||||||||||||||
| A pre-signed URL is a regular HTTPS link that includes a cryptographic signature verifying that access has been explicitly granted by someone with valid permissions. | ||||||||||||||||
| When a pre-signed URL is used, Apify validates the signature and grants access without requiring an API token. | ||||||||||||||||
|
|
||||||||||||||||
| The signature can be temporary (set to expire after a specified duration) or permanent, depending on the expiration date set when it's generated. | ||||||||||||||||
|
|
||||||||||||||||
| #### What links can be pre-signed | ||||||||||||||||
|
|
||||||||||||||||
| Only selected **dataset** and **key-value store** endpoints support pre-signed URLs. | ||||||||||||||||
| This allows fine-grained control over what data can be shared without authentication. | ||||||||||||||||
|
|
||||||||||||||||
| | Resource | Link | Validity | Notes | | ||||||||||||||||
| |-----------|-----------------------|------|-------| | ||||||||||||||||
| | **Datasets** | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. | | ||||||||||||||||
| | **Key-value stores** | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. | | ||||||||||||||||
| | **Key-value stores** | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | **Permanent only** | The public URL for a specific record is always permanent - it stays valid as long as the record exists. | | ||||||||||||||||
|
|
||||||||||||||||
|
||||||||||||||||
| | **Datasets** | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. | | |
| | **Key-value stores** | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. | | |
| | **Key-value stores** | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | **Permanent only** | The public URL for a specific record is always permanent - it stays valid as long as the record exists. | | |
| | _Datasets_ | [Dataset items](/api/v2/dataset-items-get) (`/v2/datasets/:datasetId/items`) | Temporary or Permanent | The link provides access to all dataset items. | | |
| | _Key-value stores_ | [List of keys](/api/v2/key-value-store-keys-get) (`/v2/key-value-stores/:storeId/keys`) | Temporary or Permanent | Returns the list of keys in a store. | | |
| | _Key-value stores_ | [Single record](/api/v2/key-value-store-record-get) (`/v2/key-value-stores/:storeId/records/:recordKey`) | _Permanent only_ | The public URL for a specific record is always permanent - it stays valid as long as the record exists. | | |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| These automatically generated URLs are **valid for 14 days**. | |
| These automatically generated URLs are _valid for 14 days_. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(general comment)
@raethlo This is essentially... you know... sharing 😄
It's just weird that we have a share modal, and then an export modal, and they both do kind of the same thing. It would totally make sense to somehow integrate it: If you want to share a storage, you should be able to generate a pre-signed link with just content from there.
I have a vague feeling that this has been already brought up somewhere... 🤔
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I agree completely, we may have talked about it too when I was writing the previous iteration of docs, I think from UX standpoint we should def combine export/sharing.
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| The link will include a signature **only if the general resource access is set to Restricted**. For unrestricted datasets, the link will work without a signature. | |
| The link will include a signature _only if the general resource access is set to Restricted_. For unrestricted datasets, the link will work without a signature. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be H5?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be H5?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be H5?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be H5?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be H5?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try to minimize emoji usage
| 👉 [See reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179) | |
| [Check the reference implementation in Apify clients](https://github.com/apify/apify-client-js/blob/5efd68a3bc78c0173a62775f79425fad78f0e6d1/src/resource_clients/dataset.ts#L179) |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Manual signing uses standard **HMAC (SHA-256)** with `urlSigningSecretKey` of the resource and can be easily integrated. | |
| Manual signing uses standard _HMAC (SHA-256)_ with `urlSigningSecretKey` of the resource and can be easily integrated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.