-
Notifications
You must be signed in to change notification settings - Fork 132
feat: General Resource Access #1894
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 20 commits
a530b98
d7ffedf
253d765
04599be
d4071ff
462d78c
86ad0af
27ceaec
40d1e16
9da3219
712e8a0
3cac3b2
ca7bc6f
978660e
4ec13d8
ee2fb73
296b40f
bda99f4
2628863
4038e7f
f713a6b
6f8d07c
d62db4d
0fd7bab
bd0f2fc
5a6249b
cbcbc98
3a0053f
490ddc2
b96ec6f
9002f6d
6934a33
fef7a2b
936607f
76b4405
f774ce3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,172 @@ | ||||||||||||||||||||||
| --- | ||||||||||||||||||||||
| title: General resource access | ||||||||||||||||||||||
| description: <TODO> | ||||||||||||||||||||||
| sidebar_position: 1 | ||||||||||||||||||||||
| category: platform | ||||||||||||||||||||||
| slug: /collaboration/general-resource-access | ||||||||||||||||||||||
| --- | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Some resources, like storages, Actor runs or Actor builds, support `unrestricted access`. You can share them simply by sending their unique resource ID or Console link and the recipient can then view the data in Console or fetch it via API without needing an API token. This is very useful for ad-hoc collaboration, integrating third party tools that connect to data in your Apify account or quick prototypes. | ||||||||||||||||||||||
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Thanks to the hard-to-guess, unique IDs, it’s also secure enough for most use cases. However, it doesn't offer features like access revocation or a formal audit trail and in some cases, you may want to have more direct control over data access and require users to have explicit permissions to your resources. | ||||||||||||||||||||||
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| _General resource access_ is an account setting that defines the default access policy at the account level. You can find general resource access in Apify Console under **Settings → Security & Privacy**. The two following options are supported: | ||||||||||||||||||||||
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - **Anyone with ID can read (default)**: Selected resources can be accessed using just with their unique resource ID. This means if you share the resource ID with someone, they would be able to view it without providing an API token or viewing the resource by visiting the Console URL. | ||||||||||||||||||||||
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||||||
| - **Restricted**: With this setting, only signed-in users with an explicit access to the resources can access them. To access restricted resources via API, a valid token needs to be provided. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| This setting affects the following resources: | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| - Actor runs | ||||||||||||||||||||||
| - Actor builds | ||||||||||||||||||||||
| - Storages: | ||||||||||||||||||||||
| - Datasets | ||||||||||||||||||||||
| - Key-value stores | ||||||||||||||||||||||
| - Request queues | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| Access to resources that require explicit access — such as Actors, tasks or schedules are not affected by this setting. | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
|  | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| ## How restricted access works | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| If your **general resource access** is set to **anyone with ID can read**, you can just send this link to anybody, and they will be able to download the data even if they don’t have an Apify account. However, once you change the setting to **restricted**, this API call will require a valid token with access in order to work. In other words, you’ll have to explicitly share the dataset and you can only do that with people who have an Apify account. | ||||||||||||||||||||||
|
||||||||||||||||||||||
| If your **general resource access** is set to **anyone with ID can read**, you can just send this link to anybody, and they will be able to download the data even if they don’t have an Apify account. However, once you change the setting to **restricted**, this API call will require a valid token with access in order to work. In other words, you’ll have to explicitly share the dataset and you can only do that with people who have an Apify account. | |
| If your **General resource access** is set to **Anyone with ID can read**, you can just send this link to anybody, and they will be able to download the data even if they don’t have an Apify account. However, once you change the setting to **Restricted**, this API call will require a valid token with access in order to work. In other words, you’ll have to explicitly share the dataset and you can only do that with people who have an Apify account. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why Restricted here with code notation while in other places with bold? Any particular reason?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(nit) This paragraph seems to repeat the same information as the previous one. Maybe we can just drop it?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question about formatting consistency?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(opt): maybe add a link to section where it is explained?
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This a brand new feature, that we should actually promote separately. People might like it.
I would just pull it to a top-level section and call it something like Share storages by name. Or move it under Per-resource access control
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
each admonition should have a brief title that allows users to scan the text
| :::tip | |
| This is very useful if you wish to expose a storage publicly with an easy to remember URL. | |
| ::: | |
| :::tip | |
| This is very useful if you wish to expose a storage publicly with an easy to remember URL. | |
| ::: |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would change the structure a bit:
- I would put these into a separate top-level section, as Michal suggested, so that you don't have to repeat
Exception in every header. - I would move the section to the bottom of the doc. It's IMHO not that important. In
How restricted access worksI would just say at the end that there are exception and see the sections below.
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| This ensures that public Actors in the Apify Store continue to work as expected. For example, if you open a public Actor in the Console, you’ll also be able to view its build details, download logs, or inspect the source package — without needing extra permissions or a token. | |
| This ensures that public Actors in the Apify Store continue to work as expected. For example, if you open a public Actor in Console, you’ll also be able to view its build details, download logs, or inspect the source package — without needing extra permissions or a token. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, might have missed that, thanks for catching, so the wording should be "when you share an Actor you may share list of your runs with them..." Will fix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm missing here the explanation of how access to default run storages work. That one is extremely confusing and should be definitely covered 😄
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we shouldn't have admonition next to other admonition, this is just visual overload
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Resource objects returned by the API and clients (like `apify-client-js`) include a `consoleUrl` property. This provides a stable link to the resource's page in the Apify Console. Unlike a direct API link, the Console link will prompt unauthenticated users to sign in, ensuring they have required permissions to view the resource. | |
| Resource objects returned by the API and clients (like `apify-client-js`) include a `consoleUrl` property. This provides a stable link to the resource's page in the Apify Console. Unlike a direct API link, Console link will prompt unauthenticated users to sign in, ensuring they have required permissions to view the resource. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing title for admonition
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No punctuation signs in headings
| ### What is the best setting for me? | |
| ### What is the best setting for me |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would make this a top level section and move it under How restricted access works.
This applies to the account setting, so IMHO it does not make much sense to keep it under Per-resource access control
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consistency in naming a feature, shouldn't it be uppercase since that is how it is seen in the UI? And again with formatting inconsistency, I recommend bold, but in other places we also have code notation using backticks
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consistency in naming a feature, shouldn't it be uppercase since that is how it is seen in the UI? And again with formatting inconsistency, I recommend bold, but in other places we also have code notation using backticks
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing title
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also extend it with specific instructions how to push KVS record URLs to dataset so that it renders properly in Console. This is a common problem. But this can definitely wait for later.
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consistency in naming a feature, shouldn't it be uppercase since that is how it is seen in the UI? And again with formatting inconsistency, I recommend bold, but in other places we also have code notation using backticks
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missing admonition title
Feature referred to in lower-case when on UI it's upper
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing admonition title
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -9,14 +9,28 @@ slug: /collaboration | |||||
| **Learn how to collaborate with other users and manage permissions for organizations or private resources such as Actors, Actor runs, and storages.** | ||||||
|
|
||||||
| --- | ||||||
| Apify was built from the ground up as a collaborative platform. Whether you’re publishing your Actor in the Apify Store or sharing a dataset with a teammate, collaboration is deeply integrated into how Apify works. You can easily share your resources (like Actors, runs, or storages) with others, manage permissions, or invite collaborators to your organization. By default, each system resource you create is only available to you, the owner. However, you can grant access to other users, making it easy to collaborate effectively and securely. | ||||||
|
||||||
| Apify was built from the ground up as a collaborative platform. Whether you’re publishing your Actor in the Apify Store or sharing a dataset with a teammate, collaboration is deeply integrated into how Apify works. You can easily share your resources (like Actors, runs, or storages) with others, manage permissions, or invite collaborators to your organization. By default, each system resource you create is only available to you, the owner. However, you can grant access to other users, making it easy to collaborate effectively and securely. | |
| Apify was built from the ground up as a collaborative platform. Whether you’re publishing your Actor in Apify Store or sharing a dataset with a teammate, collaboration is deeply integrated into how Apify works. You can share your resources (like Actors, runs, or storages) with others, manage permissions, or invite collaborators to your organization. By default, each system resource you create is only available to you, the owner. However, you can grant access to other users, making it easy to collaborate effectively and securely. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| While most resources can be shared by assigning permissions (see [Access Rights](./access_rights.md)), some resources can also be shared simply by using their unique links or IDs. There are two types of resources in terms of sharing: | |
| While most resources can be shared by assigning permissions (check out [Access Rights documentation](./access_rights.md)), some resources can also be shared simply by using their unique links or IDs. There are two types of resources in terms of sharing: |
raethlo marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - **resources supporting both explicit access and link sharing:** | |
| - _Resources supporting both explicit access and link sharing:_ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -558,6 +558,10 @@ A detailed tutorial on how to process one request queue with multiple Actor runs | |
|
|
||
| You can grant [access rights](../collaboration/index.md) to your request queue through the **Share** button under the **Actions** menu. For more details check the [full list of permissions](../collaboration/list_of_permissions.md). | ||
|
|
||
| You can also share request queues by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we actually support this on request queues? I'm not sure 😄 @danpoletaev ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we have generic share modal for all storages 🙃 |
||
|
|
||
| For one-off sharing of specific records when access is restricted, you can generate time-limited pre-signed URLs. See [Sharing restricted resources with pre-signed URLs](/platform/collaboration/general-resource-access#pre-signed-urls). | ||
|
|
||
| ### Sharing request queues between runs | ||
|
|
||
| You can access a request queue from any [Actor](../actors/index.mdx) or [task](../actors/running/tasks.md) run as long as you know its _name_ or _ID_. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -160,7 +160,17 @@ For example, storage names `janedoe~my-storage-1` and `janedoe~web-scrape-result | |
|
|
||
| ## Sharing | ||
|
|
||
| You can grant [access rights](../collaboration/index.md) to others Apify users to view or modify your storages. Check the [full list of permissions](../collaboration/list_of_permissions.md). | ||
| You can grant [access rights](../collaboration/index.md) to other Apify users to view or modify your storages. Check the [full list of permissions](../collaboration/list_of_permissions.md). | ||
|
|
||
| You can also share storages by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). | ||
|
|
||
| For one-off sharing when access is restricted, generate time-limited pre-signed URLs. See [Sharing restricted resources with pre-signed URLs](/platform/collaboration/general-resource-access#pre-signed-urls). | ||
|
|
||
| :::tip | ||
|
|
||
| If your storage resource is set to _restricted_, all API calls must include a valid authentication token in the `Authorization` header. If you're using **apify-client** the header is passed in automatically. | ||
|
|
||
| ::: | ||
|
|
||
| ### Sharing storages between runs | ||
|
|
||
|
|
@@ -173,6 +183,13 @@ Storage can be accessed from any [Actor](../actors/index.mdx) or [task](../actor | |
| > When multiple runs try to write data to a storage simultaneously, the order of data writing cannot be controlled. Data is written as each request is processed. <br/> | ||
| > Similar principle applies in key-value stores and request queues, when a delete request for a record precedes a read request for the same record, the read request will fail. | ||
|
|
||
| :::info | ||
|
|
||
| If a storage resource access is set to `Restricted`, you must the run from which it's accessed must have explicit access to it. | ||
| Learn how restricted access works in [General resource access](/platform/collaboration/general-resource-access). | ||
|
|
||
| ::: | ||
|
||
|
|
||
| ## Deleting storages | ||
|
|
||
| Named storages are only removed upon your request.<br/> | ||
|
|
||


There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Documentation Placeholder Not Updated
The
descriptionfield insources/platform/collaboration/general-resource-access.mdstill contains the<TODO>placeholder instead of its intended content.