You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: sources/academy/platform/actorization_playbook.mdx
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -58,14 +58,14 @@ You can Actorize various projects ranging from open-source libraries, throughout
58
58
59
59
Use these criteria to decide if your project is a good candidate for Actorization:
60
60
61
-
1._Is it self-contained?_ Does the project work non-interactively, with a well-defined, preferably structured input and output format? Positive examples include various data processing utilities, web scrapers and other automation scripts. Negative examples are GUI applications or applications that run indefinitely. If you want to run HTTP APIs on Apify, you can do so using [Actor Standby](/platform/actors/development/programming-interface/standby).
62
-
2._Can the state be stored in Apify storages?_ If the application has state that can be stored in a small number of files it can utilize [key-value store](/platform/storage/key-value-store), or if it processes records that can be stored in Apify’s [request queue](/platform/storage/request-queue). If the output consists of one or many similar JSON objects, it can utilize [dataset](/platform/storage/dataset).
61
+
1._Is it self-contained?_ Does the project work non-interactively, with a well-defined, preferably structured input and output format? Positive examples include various data processing utilities, web scrapers and other automation scripts. Negative examples are GUI applications or applications that run indefinitely. If you want to run HTTP APIs on Apify, you can do so using [Actor Standby](/platform/using-actors/development/programming-interface/standby).
62
+
2._Can the state be stored in Apify storages?_ If the application has state that can be stored in a small number of files it can utilize [key-value store](/platform/core-concepts/storage/key-value-store), or if it processes records that can be stored in Apify’s [request queue](/platform/core-concepts/storage/request-queue). If the output consists of one or many similar JSON objects, it can utilize [dataset](/platform/core-concepts/storage/dataset).
63
63
3._Can it be containerized?_ The project needs to be able to run in a Docker container. Apify currently does not support GPU workloads. External services (e.g., databases) need to be managed by developer.
64
64
4._Can it use Apify tooling?_ Javascript/Typescript applications and Python applications can be Actorized with the help of the [Apify SDK](/sdk), which makes easy for your code to interacts with the Apify platform. Applications that can be run using just the CLI can also be Actorized using the Apify CLI by writing a simple shell script that retrieves user input using [Apify CLI](/cli/), then runs your application and sends the results back to Apify (also using the CLI). If your application is implemented differently, you can still call the [Apify API](/api/v2) directly - it’s just HTTP and pretty much every language has support for that but the implementation is less straightforward.
65
65
66
66
## Actorization guide
67
67
68
-
This guide outlines the steps to convert your application into an Apify [Actor](/platform/actors). Follow the documentation links for detailed information - this guide provides an overview rather than exhaustive instructions.
68
+
This guide outlines the steps to convert your application into an Apify [Actor](/platform/using-actors). Follow the documentation links for detailed information - this guide provides an overview rather than exhaustive instructions.
69
69
70
70
### 1. Add Actor metadata - the `.actor` folder
71
71
@@ -88,9 +88,9 @@ In case you're starting a new project, we strongly advise to start with a [templ
88
88
:::
89
89
90
90
91
-
The newly created `.actor` folder contains an `actor.json` file - a manifest of the Actor. See [documentation](/platform/actors/development/actor-definition/actor-json) for more details
91
+
The newly created `.actor` folder contains an `actor.json` file - a manifest of the Actor. See [documentation](/platform/using-actors/development/actor-definition/actor-json) for more details
92
92
93
-
You must also make sure your Actor has a Dockerfile and that it installs everything needed to successfully run your application. Check out [Dockerfile documentation](/platform/actors/development/actor-definition/dockerfile) by Apify. If you don't want to use these, you are free to use any image as the base of your Actor.
93
+
You must also make sure your Actor has a Dockerfile and that it installs everything needed to successfully run your application. Check out [Dockerfile documentation](/platform/using-actors/development/actor-definition/dockerfile) by Apify. If you don't want to use these, you are free to use any image as the base of your Actor.
94
94
95
95
When launching the Actor, the Apify platform will simply run your Docker image. This means that a) you need to configure the `ENTRYPOINT` and `CMD` directives so that it launches your application and b) you can test your image locally using Docker.
96
96
@@ -100,7 +100,7 @@ These steps are the bare minimum you need to run your code on Apify. The rest of
100
100
101
101
Most Actors accept an input and produce an output. As part of Actorization, you need to define the input and output structure of your application.
102
102
103
-
For detailed information, read the docs for [input schema](/platform/actors/development/actor-definition/input-schema), [dataset schema](/platform/actors/development/actor-definition/dataset-schema), and general [storage](/platform/storage).
103
+
For detailed information, read the docs for [input schema](/platform/using-actors/development/actor-definition/input-schema), [dataset schema](/platform/using-actors/development/actor-definition/dataset-schema), and general [storage](/platform/core-concepts/storage).
104
104
105
105
106
106
#### Design guidelines
@@ -113,11 +113,11 @@ For detailed information, read the docs for [input schema](/platform/actors/deve
113
113
114
114
### 3. Handle state persistence (optional)
115
115
116
-
If your application performs a number of well-defined subtasks, the [request queue](/platform/storage/request-queue) lets you pause and resume execution on job restart. This is important for long-running jobs that might be migrated between servers at some point. In addition, this allows the Apify platform to display the progress to your users in the UI.
116
+
If your application performs a number of well-defined subtasks, the [request queue](/platform/core-concepts/storage/request-queue) lets you pause and resume execution on job restart. This is important for long-running jobs that might be migrated between servers at some point. In addition, this allows the Apify platform to display the progress to your users in the UI.
117
117
118
118
A lightweight alternative to the request queue is simply storing the state of your application as a JSON object in the key-value store and checking for that when your Actor is starting.
119
119
120
-
Fully-fledged Actors will often combine these two approaches for maximum reliability. More on this topic you find in the [state persistence](/platform/actors/development/builds-and-runs/state-persistence) article.
120
+
Fully-fledged Actors will often combine these two approaches for maximum reliability. More on this topic you find in the [state persistence](/platform/using-actors/development/builds-and-runs/state-persistence) article.
121
121
122
122
### 4. Write Actorization code
123
123
@@ -127,12 +127,12 @@ Unless you’re writing an application targeted directly on the Apify platform,
127
127
128
128
Apify provides SDKs for [Javascript](/sdk/js/) and [Python](/sdk/python/) plus a [Apify CLI](/cli/) allowing an easy interaction with Apify platform from command line.
129
129
130
-
Check out [programming interface](/platform/actors/development/programming-interface/) documentation article for details on interacting with the Apify platform in your Actor's code.
130
+
Check out [programming interface](/platform/using-actors/development/programming-interface/) documentation article for details on interacting with the Apify platform in your Actor's code.
131
131
132
132
### 5. Deploy the Actor
133
133
134
-
Deployment to Apify platform can be done easily via `apify push` command of [Apify CLI](/cli/) and for details see [deployment](/platform/actors/development/deployment) documentation.
134
+
Deployment to Apify platform can be done easily via `apify push` command of [Apify CLI](/cli/) and for details see [deployment](/platform/using-actors/development/deployment) documentation.
135
135
136
136
### 6. Publish and monetize
137
137
138
-
For details on publishing the Actor in [Apify Store](https://apify.com/store) see the [Publishing and monetization](/platform/actors/publishing). You can also follow our guide on [How to create an Actor README](/academy/actor-marketing-playbook/actor-basics/how-to-create-an-actor-readme) and [Actor marketing playbook](/academy/actor-marketing-playbook).
138
+
For details on publishing the Actor in [Apify Store](https://apify.com/store) see the [Publishing and monetization](/platform/publishing-and-monetization/publish). You can also follow our guide on [How to create an Actor README](/academy/actor-marketing-playbook/actor-basics/how-to-create-an-actor-readme) and [Actor marketing playbook](/academy/actor-marketing-playbook).
Copy file name to clipboardExpand all lines: sources/academy/platform/deploying_your_code/deploying.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -63,4 +63,4 @@ The next step is to test your Actor and experiment with the vast amount of featu
63
63
64
64
## Wrap up {#next}
65
65
66
-
That's it! In this short section, you've learned how to take your code written in any programming language and turn it into a usable Actor that can run on the Apify platform! The next step is to start looking into the [paid Actors](/platform/actors/publishing) program, which allows you to monetize your work.
66
+
That's it! In this short section, you've learned how to take your code written in any programming language and turn it into a usable Actor that can run on the Apify platform! The next step is to start looking into the [paid Actors](/platform/publishing-and-monetization/publish) program, which allows you to monetize your work.
Though writing an [input schema](/platform/actors/development/actor-definition/input-schema) for an Actor is not a required step, it is most definitely an ideal one. The Apify platform will read the **INPUT_SCHEMA.json** file within the root of your project and generate a user interface for entering input into your Actor, which makes it significantly easier for non-developers (and even developers) to configure and understand the inputs your Actor can receive. Because of this, we'll be writing an input schema for our example Actor.
12
+
Though writing an [input schema](/platform/using-actors/development/actor-definition/input-schema) for an Actor is not a required step, it is most definitely an ideal one. The Apify platform will read the **INPUT_SCHEMA.json** file within the root of your project and generate a user interface for entering input into your Actor, which makes it significantly easier for non-developers (and even developers) to configure and understand the inputs your Actor can receive. Because of this, we'll be writing an input schema for our example Actor.
13
13
14
14
> Without an input schema, the users of our Actor will have to provide the input in JSON format, which can be problematic for those who are not familiar with JSON.
15
15
@@ -51,7 +51,7 @@ Each property's key corresponds to the name we're expecting within our code, whi
Within our new **numbers** property, there are two more fields we must specify. Firstly, we must let the platform know that we're expecting an array of numbers with the **type** field. Then, we should also instruct Apify on which UI component to render for this input property. In our case, we have an array of numbers, which means we should use the **json** editor type that we discovered in the ["array" section](/platform/actors/development/actor-definition/input-schema/specification/v1#array) of the input schema documentation. We could also use **stringList**, but then we'd have to parse out the numbers from the strings.
54
+
Within our new **numbers** property, there are two more fields we must specify. Firstly, we must let the platform know that we're expecting an array of numbers with the **type** field. Then, we should also instruct Apify on which UI component to render for this input property. In our case, we have an array of numbers, which means we should use the **json** editor type that we discovered in the ["array" section](/platform/using-actors/development/actor-definition/input-schema/specification/v1#array) of the input schema documentation. We could also use **stringList**, but then we'd have to parse out the numbers from the strings.
55
55
56
56
```json
57
57
{
@@ -102,8 +102,8 @@ Here is what the input schema we wrote will render on the platform:
102
102
103
103
Later on, we'll be building more complex input schemas, as well as discussing how to write quality input schemas that allow the user to understand the Actor and not become overwhelmed.
104
104
105
-
It's not expected to memorize all of the fields that properties can take or the different editor types available, which is why it's always good to reference the [input schema documentation](/platform/actors/development/actor-definition/input-schema) when writing a schema.
105
+
It's not expected to memorize all of the fields that properties can take or the different editor types available, which is why it's always good to reference the [input schema documentation](/platform/using-actors/development/actor-definition/input-schema) when writing a schema.
106
106
107
107
## Next up {#next}
108
108
109
-
In the [next lesson](/platform/actors/development/actor-definition/dataset-schema), we'll learn how to generate an appealing Overview table to display our Actor's results in real time, so users can get immediate feedback about the data being extracted.
109
+
In the [next lesson](/platform/using-actors/development/actor-definition/dataset-schema), we'll learn how to generate an appealing Overview table to display our Actor's results in real time, so users can get immediate feedback about the data being extracted.
Copy file name to clipboardExpand all lines: sources/academy/platform/deploying_your_code/inputs_outputs.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,7 +19,7 @@ An important thing to understand regarding inputs and outputs is that they are r
19
19
20
20
## A bit about storage {#about-storage}
21
21
22
-
You can read/write your inputs/outputs: to the [key-value store](/platform/storage/key-value-store), or to the [dataset](/platform/storage/dataset). The key-value store can be used to store any sort of unorganized/unrelated data in any format, while the data pushed to a dataset typically resembles a table with columns (fields) and rows (items). Each Actor's run is allocated both a default dataset and a default key-value store.
22
+
You can read/write your inputs/outputs: to the [key-value store](/platform/core-concepts/storage/key-value-store), or to the [dataset](/platform/core-concepts/storage/dataset). The key-value store can be used to store any sort of unorganized/unrelated data in any format, while the data pushed to a dataset typically resembles a table with columns (fields) and rows (items). Each Actor's run is allocated both a default dataset and a default key-value store.
23
23
24
24
When running locally, these storages are accessible through the **storage** folder within your project's root directory, while on the platform they are accessible via Apify's API.
Copy file name to clipboardExpand all lines: sources/academy/platform/deploying_your_code/output_schema.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -160,7 +160,7 @@ const results = {
160
160
161
161
Great! Now that everything is set up, it's time to run the Actor and admire your Actor's brand new output tab.
162
162
163
-
> Need some extra guidance? Visit the [dataset schema documentation](/platform/actors/development/actor-definition/dataset-schema) for more detailed information about how to implement this feature.
163
+
> Need some extra guidance? Visit the [dataset schema documentation](/platform/using-actors/development/actor-definition/dataset-schema) for more detailed information about how to implement this feature.
164
164
165
165
A few seconds after running the Actor, you should see its results displayed in the `Overview` table.
Copy file name to clipboardExpand all lines: sources/academy/platform/expert_scraping_with_apify/migrations_maintaining_state.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,14 +19,14 @@ When a migration happens, you want to do a so-called "state transition", which m
19
19
20
20
## Learning 🧠 {#learning}
21
21
22
-
Read this [article](/platform/actors/development/builds-and-runs/state-persistence) on migrations and dealing with state transitions.
22
+
Read this [article](/platform/using-actors/development/builds-and-runs/state-persistence) on migrations and dealing with state transitions.
23
23
24
24
Before moving forward, read about Actor [events](/sdk/js/docs/upgrading/upgrading-to-v3#events) and how to listen for them.
25
25
26
26
## Knowledge check 📝 {#quiz}
27
27
28
28
1. Actors have an option in the **Settings** tab to **Restart on error**. Would you use this feature for regular Actors? When would you use this feature?
29
-
2. Migrations happen randomly, but by [aborting **gracefully**](/platform/actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?
29
+
2. Migrations happen randomly, but by [aborting **gracefully**](/platform/using-actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?
30
30
3. Why don't you (usually) need to add any special migration handling code for a standard crawling/scraping Actor? Are there any features in the Crawlee/Apify SDK that handle this under the hood?
31
31
4. How can you intercept the migration event? How much time do you have after this event happens and before the Actor migrates?
32
32
5. When would you persist data to the default key-value store instead of to a named key-value store?
Copy file name to clipboardExpand all lines: sources/academy/platform/expert_scraping_with_apify/solutions/handling_migrations.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -231,7 +231,7 @@ That's everything! Now, even if the Actor migrates (or is gracefully aborted and
231
231
232
232
**A:** It's not best to use this option by default. If it fails, there must be a reason, which would need to be thought through first - meaning that the edge case of failing should be handled when resurrecting the Actor. The state should be persisted beforehand.
233
233
234
-
**Q: Migrations happen randomly, but by [aborting gracefully](/platform/actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?**
234
+
**Q: Migrations happen randomly, but by [aborting gracefully](/platform/using-actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?**
235
235
236
236
**A:** After aborting or throwing an error mid-process, it manages to start back from where it was upon resurrection.
0 commit comments