diff --git a/.github/styles/config/vocabularies/Docs/accept.txt b/.github/styles/config/vocabularies/Docs/accept.txt index 59514bdd83..c2f7a265c4 100644 --- a/.github/styles/config/vocabularies/Docs/accept.txt +++ b/.github/styles/config/vocabularies/Docs/accept.txt @@ -53,7 +53,7 @@ proxied LLM embedder chatbot -[Ll]angchain +[Ll]angChain [Kk]eboola [Aa]irbyte @@ -68,7 +68,7 @@ exploitability [Ww]hitepaper [Cc]ron scalably -metamorph +[Mm]etamorph hostname IPs unscoped diff --git a/nginx.conf b/nginx.conf index d6dd839ab0..457e285d6e 100644 --- a/nginx.conf +++ b/nginx.conf @@ -26,6 +26,56 @@ server { return 200 '{"status":"UP"}'; } + # =================================================================== + # Platform IA Reorganization - Phase 3 Redirects + # =================================================================== + + # Using Actors (from actors/running) + rewrite ^/platform/actors/running/?$ /platform/using-actors permanent; + rewrite ^/platform/actors/running/(.+)$ /platform/using-actors/$1 permanent; + + # Core Concepts - Storage + rewrite ^/platform/storage/?$ /platform/core-concepts/storage permanent; + rewrite ^/platform/storage/(.+)$ /platform/core-concepts/storage/$1 permanent; + + # Core Concepts - Proxy + rewrite ^/platform/proxy/?$ /platform/core-concepts/proxy permanent; + rewrite ^/platform/proxy/(.+)$ /platform/core-concepts/proxy/$1 permanent; + + # Building Actors (from actors/development) + rewrite ^/platform/actors/development/?$ /platform/building-actors permanent; + rewrite ^/platform/actors/development/(.+)$ /platform/building-actors/$1 permanent; + + # Publishing & Monetization (from actors/publishing) + rewrite ^/platform/actors/publishing/?$ /platform/publishing-and-monetization permanent; + rewrite ^/platform/actors/publishing/(.+)$ /platform/publishing-and-monetization/$1 permanent; + + # =================================================================== + # Platform IA Reorganization - Phase 4 Redirects + # =================================================================== + + # Getting Started (from building-actors/quick-start) + rewrite ^/platform/building-actors/quick-start/?$ /platform/getting-started permanent; + rewrite ^/platform/building-actors/quick-start/locally/?$ /platform/getting-started/locally permanent; + rewrite ^/platform/building-actors/quick-start/web-ide/?$ /platform/getting-started/web-ide permanent; + rewrite ^/platform/building-actors/quick-start/build-with-ai/?$ /platform/getting-started/build-with-ai permanent; + + # Integrations (no path change, but ensure consistency) + # Note: Integrations already at /platform/integrations - no redirect needed + + # Collaboration + rewrite ^/platform/collaboration/?$ /platform/collaboration permanent; + rewrite ^/platform/collaboration/(.+)$ /platform/collaboration/$1 permanent; + + # Monitoring & Operations (from monitoring) + rewrite ^/platform/monitoring/?$ /platform/monitoring-and-operations/monitoring permanent; + rewrite ^/platform/monitoring/(.+)$ /platform/monitoring-and-operations/monitoring/$1 permanent; + + # Reference (orphaned files) + rewrite ^/platform/limits$ /platform/reference/limits permanent; + rewrite ^/platform/security$ /platform/reference/security permanent; + rewrite ^/platform/schedules$ /platform/reference/schedules permanent; + # generated via `node tools/convert.mjs` inside the docs repository rewrite ^/about$ /platform/about permanent; rewrite ^/access-rights$ /platform/access-rights permanent; diff --git a/sources/academy/ai/ai-agents.mdx b/sources/academy/ai/ai-agents.mdx index 4e53884a28..c35fe8fe15 100644 --- a/sources/academy/ai/ai-agents.mdx +++ b/sources/academy/ai/ai-agents.mdx @@ -52,7 +52,7 @@ _Example input:_ _Example output:_ - Text response with insights -- Data stored in Apify [Dataset](/platform/storage/dataset) +- Data stored in Apify [Dataset](/platform/core-concepts/storage/dataset) :::note Agent memory diff --git a/sources/academy/glossary/tools/apify_cli.md b/sources/academy/glossary/tools/apify_cli.md index 82cb187e77..26d29d06b0 100644 --- a/sources/academy/glossary/tools/apify_cli.md +++ b/sources/academy/glossary/tools/apify_cli.md @@ -15,7 +15,7 @@ The [Apify CLI](/cli) helps you create, develop, build and run Apify Actors, and ## Installing {#installing} -To install the Apfiy CLI, you'll first need npm, which comes preinstalled with Node.js. If you haven't yet installed Node, learn how to do that [here](../../webscraping/scraping_basics_javascript/data_extraction/computer_preparation.md). Additionally, make sure you've got an Apify account, as you will need to log in to the CLI to gain access to its full potential. +To install the Apfiy CLI, you'll first need npm, which comes preinstalled with Node.js. If you haven't yet installed Node, [learn how to prepare your computer](../../webscraping/scraping_basics_javascript/data_extraction/computer_preparation.md). Additionally, make sure you've got an Apify account, as you will need to log in to the CLI to gain access to its full potential. Open up a terminal instance and run the following command: diff --git a/sources/academy/platform/actorization_playbook.mdx b/sources/academy/platform/actorization_playbook.mdx index d846c380f6..635d0b8f74 100644 --- a/sources/academy/platform/actorization_playbook.mdx +++ b/sources/academy/platform/actorization_playbook.mdx @@ -58,14 +58,14 @@ You can Actorize various projects ranging from open-source libraries, throughout Use these criteria to decide if your project is a good candidate for Actorization: -1. _Is it self-contained?_ Does the project work non-interactively, with a well-defined, preferably structured input and output format? Positive examples include various data processing utilities, web scrapers and other automation scripts. Negative examples are GUI applications or applications that run indefinitely. If you want to run HTTP APIs on Apify, you can do so using [Actor Standby](/platform/actors/development/programming-interface/standby). -2. _Can the state be stored in Apify storages?_ If the application has state that can be stored in a small number of files it can utilize [key-value store](/platform/storage/key-value-store), or if it processes records that can be stored in Apify’s [request queue](/platform/storage/request-queue). If the output consists of one or many similar JSON objects, it can utilize [dataset](/platform/storage/dataset). +1. _Is it self-contained?_ Does the project work non-interactively, with a well-defined, preferably structured input and output format? Positive examples include various data processing utilities, web scrapers and other automation scripts. Negative examples are GUI applications or applications that run indefinitely. If you want to run HTTP APIs on Apify, you can do so using [Actor Standby](/platform/using-actors/development/programming-interface/standby). +2. _Can the state be stored in Apify storages?_ If the application has state that can be stored in a small number of files it can utilize [key-value store](/platform/core-concepts/storage/key-value-store), or if it processes records that can be stored in Apify’s [request queue](/platform/core-concepts/storage/request-queue). If the output consists of one or many similar JSON objects, it can utilize [dataset](/platform/core-concepts/storage/dataset). 3. _Can it be containerized?_ The project needs to be able to run in a Docker container. Apify currently does not support GPU workloads. External services (e.g., databases) need to be managed by developer. 4. _Can it use Apify tooling?_ Javascript/Typescript applications and Python applications can be Actorized with the help of the [Apify SDK](/sdk), which makes easy for your code to interacts with the Apify platform. Applications that can be run using just the CLI can also be Actorized using the Apify CLI by writing a simple shell script that retrieves user input using [Apify CLI](/cli/), then runs your application and sends the results back to Apify (also using the CLI). If your application is implemented differently, you can still call the [Apify API](/api/v2) directly - it’s just HTTP and pretty much every language has support for that but the implementation is less straightforward. ## Actorization guide -This guide outlines the steps to convert your application into an Apify [Actor](/platform/actors). Follow the documentation links for detailed information - this guide provides an overview rather than exhaustive instructions. +This guide outlines the steps to convert your application into an Apify [Actor](/platform/using-actors). Follow the documentation links for detailed information - this guide provides an overview rather than exhaustive instructions. ### 1. Add Actor metadata - the `.actor` folder @@ -83,14 +83,14 @@ In case you're starting a new project, we strongly advise to start with a [templ :::note Quick Start for beginners - For a step-by-step introduction to creating your first Actor (including tech stack choices and development paths), see [Quick Start](/platform/actors/development/quick-start). + For a step-by-step introduction to creating your first Actor (including tech stack choices and development paths), see [Getting started](/platform/getting-started). ::: -The newly created `.actor` folder contains an `actor.json` file - a manifest of the Actor. See [documentation](/platform/actors/development/actor-definition/actor-json) for more details +The newly created `.actor` folder contains an `actor.json` file - a manifest of the Actor. See [documentation](/platform/using-actors/development/actor-definition/actor-json) for more details -You must also make sure your Actor has a Dockerfile and that it installs everything needed to successfully run your application. Check out [Dockerfile documentation](/platform/actors/development/actor-definition/dockerfile) by Apify. If you don't want to use these, you are free to use any image as the base of your Actor. +You must also make sure your Actor has a Dockerfile and that it installs everything needed to successfully run your application. Check out [Dockerfile documentation](/platform/using-actors/development/actor-definition/dockerfile) by Apify. If you don't want to use these, you are free to use any image as the base of your Actor. When launching the Actor, the Apify platform will simply run your Docker image. This means that a) you need to configure the `ENTRYPOINT` and `CMD` directives so that it launches your application and b) you can test your image locally using Docker. @@ -100,7 +100,7 @@ These steps are the bare minimum you need to run your code on Apify. The rest of Most Actors accept an input and produce an output. As part of Actorization, you need to define the input and output structure of your application. -For detailed information, read the docs for [input schema](/platform/actors/development/actor-definition/input-schema), [dataset schema](/platform/actors/development/actor-definition/dataset-schema), and general [storage](/platform/storage). +For detailed information, read the docs for [input schema](/platform/using-actors/development/actor-definition/input-schema), [dataset schema](/platform/using-actors/development/actor-definition/dataset-schema), and general [storage](/platform/core-concepts/storage). #### Design guidelines @@ -113,11 +113,11 @@ For detailed information, read the docs for [input schema](/platform/actors/deve ### 3. Handle state persistence (optional) -If your application performs a number of well-defined subtasks, the [request queue](/platform/storage/request-queue) lets you pause and resume execution on job restart. This is important for long-running jobs that might be migrated between servers at some point. In addition, this allows the Apify platform to display the progress to your users in the UI. +If your application performs a number of well-defined subtasks, the [request queue](/platform/core-concepts/storage/request-queue) lets you pause and resume execution on job restart. This is important for long-running jobs that might be migrated between servers at some point. In addition, this allows the Apify platform to display the progress to your users in the UI. A lightweight alternative to the request queue is simply storing the state of your application as a JSON object in the key-value store and checking for that when your Actor is starting. -Fully-fledged Actors will often combine these two approaches for maximum reliability. More on this topic you find in the [state persistence](/platform/actors/development/builds-and-runs/state-persistence) article. +Fully-fledged Actors will often combine these two approaches for maximum reliability. More on this topic you find in the [state persistence](/platform/using-actors/development/builds-and-runs/state-persistence) article. ### 4. Write Actorization code @@ -127,12 +127,12 @@ Unless you’re writing an application targeted directly on the Apify platform, Apify provides SDKs for [Javascript](/sdk/js/) and [Python](/sdk/python/) plus a [Apify CLI](/cli/) allowing an easy interaction with Apify platform from command line. -Check out [programming interface](/platform/actors/development/programming-interface/) documentation article for details on interacting with the Apify platform in your Actor's code. +Check out [programming interface](/platform/using-actors/development/programming-interface/) documentation article for details on interacting with the Apify platform in your Actor's code. ### 5. Deploy the Actor -Deployment to Apify platform can be done easily via `apify push` command of [Apify CLI](/cli/) and for details see [deployment](/platform/actors/development/deployment) documentation. +Deployment to Apify platform can be done easily via `apify push` command of [Apify CLI](/cli/) and for details see [deployment](/platform/using-actors/development/deployment) documentation. ### 6. Publish and monetize -For details on publishing the Actor in [Apify Store](https://apify.com/store) see the [Publishing and monetization](/platform/actors/publishing). You can also follow our guide on [How to create an Actor README](/academy/actor-marketing-playbook/actor-basics/how-to-create-an-actor-readme) and [Actor marketing playbook](/academy/actor-marketing-playbook). +For details on publishing the Actor in [Apify Store](https://apify.com/store) see the [Publishing and monetization](/platform/publishing-and-monetization/publish). You can also follow our guide on [How to create an Actor README](/academy/actor-marketing-playbook/actor-basics/how-to-create-an-actor-readme) and [Actor marketing playbook](/academy/actor-marketing-playbook). diff --git a/sources/academy/platform/deploying_your_code/deploying.md b/sources/academy/platform/deploying_your_code/deploying.md index 6eb41e895a..3368790d78 100644 --- a/sources/academy/platform/deploying_your_code/deploying.md +++ b/sources/academy/platform/deploying_your_code/deploying.md @@ -39,7 +39,7 @@ The final step is to click on **API** in the top right corner of our Actor's pag And scroll through all of the links until we find the **Build Actor** API endpoint. Now we'll copy this endpoint's URL, head back over to our GitHub repository and navigate to **Settings > Webhooks > Add webhook**. The final thing to do is to paste the URL and save the webhook. -![Adding a webhook to your GitHub repository](../../../platform/actors/development/deployment/images/ci-github-integration.png) +![Adding a webhook to your GitHub repository](../../../platform/5-building-actors/deployment/images/ci-github-integration.png) That's it! The Actor should now pull its source code from the repository and automatically build. @@ -63,4 +63,4 @@ The next step is to test your Actor and experiment with the vast amount of featu ## Wrap up {#next} -That's it! In this short section, you've learned how to take your code written in any programming language and turn it into a usable Actor that can run on the Apify platform! The next step is to start looking into the [paid Actors](/platform/actors/publishing) program, which allows you to monetize your work. +That's it! In this short section, you've learned how to take your code written in any programming language and turn it into a usable Actor that can run on the Apify platform! The next step is to start looking into the [paid Actors](/platform/publishing-and-monetization/publish) program, which allows you to monetize your work. diff --git a/sources/academy/platform/deploying_your_code/input_schema.md b/sources/academy/platform/deploying_your_code/input_schema.md index a8ec5e5436..9188615c75 100644 --- a/sources/academy/platform/deploying_your_code/input_schema.md +++ b/sources/academy/platform/deploying_your_code/input_schema.md @@ -9,7 +9,7 @@ slug: /deploying-your-code/input-schema --- -Though writing an [input schema](/platform/actors/development/actor-definition/input-schema) for an Actor is not a required step, it is most definitely an ideal one. The Apify platform will read the **INPUT_SCHEMA.json** file within the root of your project and generate a user interface for entering input into your Actor, which makes it significantly easier for non-developers (and even developers) to configure and understand the inputs your Actor can receive. Because of this, we'll be writing an input schema for our example Actor. +Though writing an [input schema](/platform/using-actors/development/actor-definition/input-schema) for an Actor is not a required step, it is most definitely an ideal one. The Apify platform will read the **INPUT_SCHEMA.json** file within the root of your project and generate a user interface for entering input into your Actor, which makes it significantly easier for non-developers (and even developers) to configure and understand the inputs your Actor can receive. Because of this, we'll be writing an input schema for our example Actor. > Without an input schema, the users of our Actor will have to provide the input in JSON format, which can be problematic for those who are not familiar with JSON. @@ -51,7 +51,7 @@ Each property's key corresponds to the name we're expecting within our code, whi ## Property types & editor types {#property-types} -Within our new **numbers** property, there are two more fields we must specify. Firstly, we must let the platform know that we're expecting an array of numbers with the **type** field. Then, we should also instruct Apify on which UI component to render for this input property. In our case, we have an array of numbers, which means we should use the **json** editor type that we discovered in the ["array" section](/platform/actors/development/actor-definition/input-schema/specification/v1#array) of the input schema documentation. We could also use **stringList**, but then we'd have to parse out the numbers from the strings. +Within our new **numbers** property, there are two more fields we must specify. Firstly, we must let the platform know that we're expecting an array of numbers with the **type** field. Then, we should also instruct Apify on which UI component to render for this input property. In our case, we have an array of numbers, which means we should use the **json** editor type that we discovered in the ["array" section](/platform/using-actors/development/actor-definition/input-schema/specification/v1#array) of the input schema documentation. We could also use **stringList**, but then we'd have to parse out the numbers from the strings. ```json { @@ -102,8 +102,8 @@ Here is what the input schema we wrote will render on the platform: Later on, we'll be building more complex input schemas, as well as discussing how to write quality input schemas that allow the user to understand the Actor and not become overwhelmed. -It's not expected to memorize all of the fields that properties can take or the different editor types available, which is why it's always good to reference the [input schema documentation](/platform/actors/development/actor-definition/input-schema) when writing a schema. +It's not expected to memorize all of the fields that properties can take or the different editor types available, which is why it's always good to reference the [input schema documentation](/platform/using-actors/development/actor-definition/input-schema) when writing a schema. ## Next up {#next} -In the [next lesson](/platform/actors/development/actor-definition/dataset-schema), we'll learn how to generate an appealing Overview table to display our Actor's results in real time, so users can get immediate feedback about the data being extracted. +In the [next lesson](/platform/using-actors/development/actor-definition/dataset-schema), we'll learn how to generate an appealing Overview table to display our Actor's results in real time, so users can get immediate feedback about the data being extracted. diff --git a/sources/academy/platform/deploying_your_code/inputs_outputs.md b/sources/academy/platform/deploying_your_code/inputs_outputs.md index a41f8ab377..083448aba4 100644 --- a/sources/academy/platform/deploying_your_code/inputs_outputs.md +++ b/sources/academy/platform/deploying_your_code/inputs_outputs.md @@ -19,7 +19,7 @@ An important thing to understand regarding inputs and outputs is that they are r ## A bit about storage {#about-storage} -You can read/write your inputs/outputs: to the [key-value store](/platform/storage/key-value-store), or to the [dataset](/platform/storage/dataset). The key-value store can be used to store any sort of unorganized/unrelated data in any format, while the data pushed to a dataset typically resembles a table with columns (fields) and rows (items). Each Actor's run is allocated both a default dataset and a default key-value store. +You can read/write your inputs/outputs: to the [key-value store](/platform/core-concepts/storage/key-value-store), or to the [dataset](/platform/core-concepts/storage/dataset). The key-value store can be used to store any sort of unorganized/unrelated data in any format, while the data pushed to a dataset typically resembles a table with columns (fields) and rows (items). Each Actor's run is allocated both a default dataset and a default key-value store. When running locally, these storages are accessible through the **storage** folder within your project's root directory, while on the platform they are accessible via Apify's API. diff --git a/sources/academy/platform/deploying_your_code/output_schema.md b/sources/academy/platform/deploying_your_code/output_schema.md index a18f7591a9..b3eef24f35 100644 --- a/sources/academy/platform/deploying_your_code/output_schema.md +++ b/sources/academy/platform/deploying_your_code/output_schema.md @@ -11,7 +11,7 @@ slug: /deploying-your-code/dataset-schema The dataset schema generates an interface that enables users to instantly preview their Actor results in real time. -![Dataset Schema](../../../platform/actors/development/actor_definition/images/output-schema-example.png) +![Dataset Schema](../../../platform/5-building-actors/actor_definition/images/output-schema-example.png) In this quick tutorial, you will learn how to set up an output tab for your own Actor. @@ -160,7 +160,7 @@ const results = { Great! Now that everything is set up, it's time to run the Actor and admire your Actor's brand new output tab. -> Need some extra guidance? Visit the [dataset schema documentation](/platform/actors/development/actor-definition/dataset-schema) for more detailed information about how to implement this feature. +> Need some extra guidance? Visit the [dataset schema documentation](/platform/using-actors/development/actor-definition/dataset-schema) for more detailed information about how to implement this feature. A few seconds after running the Actor, you should see its results displayed in the `Overview` table. diff --git a/sources/academy/platform/expert_scraping_with_apify/actors_webhooks.md b/sources/academy/platform/expert_scraping_with_apify/actors_webhooks.md index 53814c0033..897b55c460 100644 --- a/sources/academy/platform/expert_scraping_with_apify/actors_webhooks.md +++ b/sources/academy/platform/expert_scraping_with_apify/actors_webhooks.md @@ -15,7 +15,7 @@ Thus far, you've run Actors on the platform and written an Actor of your own, wh ## Advanced Actor overview {#advanced-actors} -In this course, we'll be working out of the Amazon scraper project from the **Web scraping basics for JavaScript devs** course. If you haven't already built that project, you can do it in three short lessons [here](../../webscraping/scraping_basics_javascript/challenge/index.md). We've made a few small modifications to the project with the Apify SDK, but 99% of the code is still the same. +In this course, we'll be working out of the Amazon scraper project from the **Web scraping basics for JavaScript devs** course. If you haven't already built that project, you can do it [in three short lessons](../../webscraping/scraping_basics_javascript/challenge/index.md). We've made a few small modifications to the project with the Apify SDK, but 99% of the code is still the same. Take another look at the files within your Amazon scraper project. You'll notice that there is a **Dockerfile**. Every single Actor has a Dockerfile (the Actor's **Image**) which tells Docker how to spin up a container on the Apify platform which can successfully run the Actor's code. "Apify Actors" is a serverless platform that runs multiple Docker containers. For a deeper understanding of Actor Dockerfiles, refer to the [Apify Actor Dockerfile docs](/sdk/js/docs/guides/docker-images#example-dockerfile). @@ -27,7 +27,7 @@ Webhooks are a powerful tool that can be used for just about anything. You can s Prior to moving forward, please read over these resources: -- Read about [running Actors, handling Actor inputs, memory and CPU](/platform/actors/running). +- Read about [running Actors, handling Actor inputs, memory and CPU](/platform/using-actors/running). - Learn about [Actor webhooks](/platform/integrations/webhooks), which we will implement in the next lesson. - Learn [how to run Actors](/academy/api/run-actor-and-retrieve-data-via-api) using Apify's REST API. diff --git a/sources/academy/platform/expert_scraping_with_apify/bypassing_anti_scraping.md b/sources/academy/platform/expert_scraping_with_apify/bypassing_anti_scraping.md index ccc9c62f3e..83bc138924 100644 --- a/sources/academy/platform/expert_scraping_with_apify/bypassing_anti_scraping.md +++ b/sources/academy/platform/expert_scraping_with_apify/bypassing_anti_scraping.md @@ -20,7 +20,7 @@ You might have already noticed that we've been using the **RESIDENTIAL** proxy g ## Learning 🧠 {#learning} - Skim [this page](https://apify.com/proxy) for a general idea of Apify Proxy. -- Give the [proxy documentation](/platform/proxy) a solid readover (feel free to skip most of the examples). +- Give the [proxy documentation](/platform/core-concepts/proxy) a solid readover (feel free to skip most of the examples). - Check out the [anti-scraping guide](../../webscraping/anti_scraping/index.md). - Gain a solid understanding of the [SessionPool](https://crawlee.dev/api/core/class/SessionPool). - Look at a few Actors on the [Apify store](https://apify.com/store). How are they utilizing proxies? diff --git a/sources/academy/platform/expert_scraping_with_apify/managing_source_code.md b/sources/academy/platform/expert_scraping_with_apify/managing_source_code.md index 38d3fdaa95..b7235f56b1 100644 --- a/sources/academy/platform/expert_scraping_with_apify/managing_source_code.md +++ b/sources/academy/platform/expert_scraping_with_apify/managing_source_code.md @@ -53,7 +53,7 @@ The final step is to click on **API** in the top right corner of your Actor's pa And scroll through all of the links until you find the **Build Actor** API endpoint. Copy this endpoint's URL, then head back over to your GitHub repository and navigate to **Settings > Webhooks > Add webhook**. The final thing to do is to paste the URL and save the webhook. -![Adding a webhook to your GitHub repo](../../../platform/actors/development/deployment/images/ci-github-integration.png) +![Adding a webhook to your GitHub repo](../../../platform/5-building-actors/deployment/images/ci-github-integration.png) And you're done! 🎉 diff --git a/sources/academy/platform/expert_scraping_with_apify/migrations_maintaining_state.md b/sources/academy/platform/expert_scraping_with_apify/migrations_maintaining_state.md index c59da80531..46fc444185 100644 --- a/sources/academy/platform/expert_scraping_with_apify/migrations_maintaining_state.md +++ b/sources/academy/platform/expert_scraping_with_apify/migrations_maintaining_state.md @@ -19,14 +19,14 @@ When a migration happens, you want to do a so-called "state transition", which m ## Learning 🧠 {#learning} -Read this [article](/platform/actors/development/builds-and-runs/state-persistence) on migrations and dealing with state transitions. +Read this [article](/platform/using-actors/development/builds-and-runs/state-persistence) on migrations and dealing with state transitions. Before moving forward, read about Actor [events](/sdk/js/docs/upgrading/upgrading-to-v3#events) and how to listen for them. ## Knowledge check 📝 {#quiz} 1. Actors have an option in the **Settings** tab to **Restart on error**. Would you use this feature for regular Actors? When would you use this feature? -2. Migrations happen randomly, but by [aborting **gracefully**](/platform/actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run? +2. Migrations happen randomly, but by [aborting **gracefully**](/platform/using-actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run? 3. Why don't you (usually) need to add any special migration handling code for a standard crawling/scraping Actor? Are there any features in the Crawlee/Apify SDK that handle this under the hood? 4. How can you intercept the migration event? How much time do you have after this event happens and before the Actor migrates? 5. When would you persist data to the default key-value store instead of to a named key-value store? diff --git a/sources/academy/platform/expert_scraping_with_apify/solutions/handling_migrations.md b/sources/academy/platform/expert_scraping_with_apify/solutions/handling_migrations.md index 635971ff65..af27b2b5c1 100644 --- a/sources/academy/platform/expert_scraping_with_apify/solutions/handling_migrations.md +++ b/sources/academy/platform/expert_scraping_with_apify/solutions/handling_migrations.md @@ -231,7 +231,7 @@ That's everything! Now, even if the Actor migrates (or is gracefully aborted and **A:** It's not best to use this option by default. If it fails, there must be a reason, which would need to be thought through first - meaning that the edge case of failing should be handled when resurrecting the Actor. The state should be persisted beforehand. -**Q: Migrations happen randomly, but by [aborting gracefully](/platform/actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?** +**Q: Migrations happen randomly, but by [aborting gracefully](/platform/using-actors/running/runs-and-builds#aborting-runs), you can simulate a similar situation. Try this out on the platform and observe what happens. What changes occur, and what remains the same for the restarted Actor's run?** **A:** After aborting or throwing an error mid-process, it manages to start back from where it was upon resurrection. diff --git a/sources/academy/platform/expert_scraping_with_apify/tasks_and_storage.md b/sources/academy/platform/expert_scraping_with_apify/tasks_and_storage.md index d18009c241..40b31ca34c 100644 --- a/sources/academy/platform/expert_scraping_with_apify/tasks_and_storage.md +++ b/sources/academy/platform/expert_scraping_with_apify/tasks_and_storage.md @@ -23,9 +23,9 @@ Storage allows us to save persistent data for further processing. As you'll lear ## Learning 🧠 {#learning} -- Check out [the docs about Actor tasks](/platform/actors/running/tasks). -- Read about the [two main storage options](/platform/storage/dataset) on the Apify platform. -- Understand the [crucial differences between named and unnamed storages](/platform/storage/usage#named-and-unnamed-storages). +- Check out [the docs about Actor tasks](/platform/using-actors/running/tasks). +- Read about the [two main storage options](/platform/core-concepts/storage/dataset) on the Apify platform. +- Understand the [crucial differences between named and unnamed storages](/platform/core-concepts/storage/usage#named-and-unnamed-storages). - Learn about the [`Dataset`](/sdk/js/reference/class/Dataset) and [`KeyValueStore`](/sdk/js/reference/class/KeyValueStore) objects in the Apify SDK. ## Knowledge check 📝 {#quiz} diff --git a/sources/academy/platform/get_most_of_actors/index.md b/sources/academy/platform/get_most_of_actors/index.md index 25c7aabbd0..b63c4a216b 100644 --- a/sources/academy/platform/get_most_of_actors/index.md +++ b/sources/academy/platform/get_most_of_actors/index.md @@ -181,4 +181,4 @@ To help you succeed, we've created a comprehensive Actor marketing playbook. You />
-Ready to grow your presence on the Apify platform? Check out our guide to [publishing your first Actor](/platform/actors/publishing). +Ready to grow your presence on the Apify platform? Check out our guide to [publishing your first Actor](/platform/publishing-and-monetization/publish). diff --git a/sources/academy/platform/get_most_of_actors/monetizing_your_actor.md b/sources/academy/platform/get_most_of_actors/monetizing_your_actor.md index 57895f76ba..08441fa143 100644 --- a/sources/academy/platform/get_most_of_actors/monetizing_your_actor.md +++ b/sources/academy/platform/get_most_of_actors/monetizing_your_actor.md @@ -72,7 +72,7 @@ For example, if you host 50GB of data for 30 days: This means that storing 50 GB of data for 30 days results in 36,000 GB-hours. ::: -Read more about Actors in the Store and different pricing models from the perspective of your users in the [Store documentation](https://docs.apify.com/platform/actors/running/actors-in-store). +Read more about Actors in the Store and different pricing models from the perspective of your users in the [Store documentation](https://docs.apify.com/platform/using-actors/running/actors-in-store).
Example - pay-per-result pricing model @@ -86,7 +86,7 @@ Your profit is computed only from the first two users, since they are on Apify p To ensure profitable operation: -- Set memory limits in your [`actor.json`](https://docs.apify.com/platform/actors/development/actor-definition/actor-json) file to control platform usage costs +- Set memory limits in your [`actor.json`](https://docs.apify.com/platform/using-actors/development/actor-definition/actor-json) file to control platform usage costs - Implement the `ACTOR_MAX_PAID_DATASET_ITEMS` check to prevent excess result generation - Test your Actor with various result volumes to determine optimal pricing diff --git a/sources/academy/platform/get_most_of_actors/product_optimization/how_to_create_a_great_input_schema.md b/sources/academy/platform/get_most_of_actors/product_optimization/how_to_create_a_great_input_schema.md index 1c9c70f1f8..7f38c098e2 100644 --- a/sources/academy/platform/get_most_of_actors/product_optimization/how_to_create_a_great_input_schema.md +++ b/sources/academy/platform/get_most_of_actors/product_optimization/how_to_create_a_great_input_schema.md @@ -24,7 +24,7 @@ In this article, _we’ll refer to the input schema as the user interface_ of yo :::tip Understand input schemas -To fully understand the recommendations in this blog post, you’ll first need to familiarize yourself with the [technical aspects of the input schema](https://docs.apify.com/platform/actors/development/actor-definition/input-schema). This context is essential to make good use of the insights shared here. +To fully understand the recommendations in this blog post, you’ll first need to familiarize yourself with the [technical aspects of the input schema](https://docs.apify.com/platform/using-actors/development/actor-definition/input-schema). This context is essential to make good use of the insights shared here. ::: @@ -161,4 +161,4 @@ The version above was the improved input schema. Here's what this tool's input s ## Resources - Basics of input schema: [https://docs.apify.com/academy/deploying-your-code/input-schema](https://docs.apify.com/academy/deploying-your-code/input-schema) -- Specifications of input schema: [https://docs.apify.com/platform/actors/development/actor-definition/input-schema](https://docs.apify.com/platform/actors/development/actor-definition/input-schema) +- Specifications of input schema: [https://docs.apify.com/platform/using-actors/development/actor-definition/input-schema](https://docs.apify.com/platform/using-actors/development/actor-definition/input-schema) diff --git a/sources/academy/platform/get_most_of_actors/store_basics/actor_success_stories.md b/sources/academy/platform/get_most_of_actors/store_basics/actor_success_stories.md index 736d12de0a..208e5bd30a 100644 --- a/sources/academy/platform/get_most_of_actors/store_basics/actor_success_stories.md +++ b/sources/academy/platform/get_most_of_actors/store_basics/actor_success_stories.md @@ -8,13 +8,13 @@ slug: /actor-marketing-playbook/store-basics/actor-success-stories _Web scraping freelance financial freedom with microworlds._ -Discover how Caleb David, founder of `microworlds`, achieved financial freedom through freelance web scraping. His journey showcases how mastering the craft with tools like Crawlee and creating a Twitter scraper transformed his career. See the full story [here](https://blog.apify.com/web-scraping-freelance-financial-freedom/) and learn from his success. +Discover how Caleb David, founder of `microworlds`, achieved financial freedom through freelance web scraping. His journey showcases how mastering the craft with tools like Crawlee and creating a Twitter scraper transformed his career. [Read the full story on the Apify blog](https://blog.apify.com/web-scraping-freelance-financial-freedom/) to learn from his success. https://apify.com/microworlds _Web scraping for freelance success – insights from Tugkan._ -In this success story, our first community dev Tugkan shares how his journey into freelancing via Apify changed his life. Learn about his process, challenges, and how his paid Actors have brought him financial rewards and freedom. Check out his story [here](https://apify.com/success-stories/paid-actor-journey-apify-freelancer-tugkan) for inspiration. +In this success story, our first community dev Tugkan shares how his journey into freelancing via Apify changed his life. Learn about his process, challenges, and how his paid Actors have brought him financial rewards and freedom. [Read Tugkan's success story](https://apify.com/success-stories/paid-actor-journey-apify-freelancer-tugkan) for inspiration. https://apify.com/epctex diff --git a/sources/academy/platform/get_most_of_actors/store_basics/how_actor_monetization_works.md b/sources/academy/platform/get_most_of_actors/store_basics/how_actor_monetization_works.md index c7e986d1df..a8a7341f7c 100644 --- a/sources/academy/platform/get_most_of_actors/store_basics/how_actor_monetization_works.md +++ b/sources/academy/platform/get_most_of_actors/store_basics/how_actor_monetization_works.md @@ -33,7 +33,7 @@ Monetizing your Actor on the Apify platform involves several key steps: - 2nd user starts their trial but pays next month. - 3rd user on a free plan finishes the trial without upgrading to a paid plan and can’t use the Actor further. -Learn more about the rental pricing model in our [documentation](/platform/actors/publishing/monetize#rental-pricing-model). +Learn more about the rental pricing model in our [documentation](/platform/publishing-and-monetization/publish/monetize#rental-pricing-model). ### Pay-per-result pricing model @@ -49,7 +49,7 @@ Learn more about the rental pricing model in our [documentation](/platform/actor - Dataset storage: $1 per 1,000 GB-hours - _Example_: you set a price of $1 per 1,000 results. Two users generate 50,000 and 20,000 results, paying $50 and $20, respectively. If the platform usage costs are $5 and $2, your profit is $49. -Learn more about the pay-per-result pricing model in our [documentation](/platform/actors/publishing/monetize#pay-per-result-pricing-model). +Learn more about the pay-per-result pricing model in our [documentation](/platform/publishing-and-monetization/publish/monetize#pay-per-result-pricing-model). ### Pay-per-event pricing model @@ -77,7 +77,7 @@ Learn more about the pay-per-result pricing model in our [documentation](/platfo - This comes up to $1.625 of total revenue - That means if platform usage costs are $0.365 for user A and $0.162 for user B your profit is $4.748 -Learn more about the pay-per-event pricing model in our [documentation](/platform/actors/publishing/monetize#pay-per-event-pricing-model). +Learn more about the pay-per-event pricing model in our [documentation](/platform/publishing-and-monetization/publish/monetize#pay-per-event-pricing-model). ## Setting up monetization @@ -150,7 +150,7 @@ Example of useful pricing estimates from the **Analytics** tab: - Learn about [incentives behind monetization](https://apify.com/partners/actor-developers) - Detailed guide to [setting up monetization models](https://docs.apify.com/academy/get-most-of-actors/monetizing-your-actor) -- Guide to [publishing Actors](https://docs.apify.com/platform/actors/publishing) +- Guide to [publishing Actors](https://docs.apify.com/platform/publishing-and-monetization/publish) - Watch our webinar on how to [build, publish and monetize Actors](https://www.youtube.com/watch?v=4nxStxC1BJM) - Read a blog post from our CEO on the [reasoning behind monetizing Actors](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) - Learn about the [Creator plan](https://apify.com/pricing/creator-plan), which allows you to create and freely test your own Actors for $1 diff --git a/sources/academy/platform/get_most_of_actors/store_basics/how_store_works.md b/sources/academy/platform/get_most_of_actors/store_basics/how_store_works.md index bcc1d7b794..ab7648d604 100644 --- a/sources/academy/platform/get_most_of_actors/store_basics/how_store_works.md +++ b/sources/academy/platform/get_most_of_actors/store_basics/how_store_works.md @@ -103,8 +103,8 @@ Learn more about how to handle the [Issues tab](/academy/actor-marketing-playboo ## Resources -- Best practices on setting up [testing for your Actor](https://docs.apify.com/platform/actors/publishing/test) +- Best practices on setting up [testing for your Actor](https://docs.apify.com/platform/publishing-and-monetization/publish/test) - What are Apify-maintained and [Community-maintained Actors](https://help.apify.com/en/articles/6999799-what-are-apify-maintained-and-community-maintained-actors)? On ownership, maintenance, features, and support -- Step-by-step guide on how to [publish your Actor](https://docs.apify.com/platform/actors/publishing) +- Step-by-step guide on how to [publish your Actor](https://docs.apify.com/platform/publishing-and-monetization/publish) - Watch our webinar on how to [build, publish and monetize Actors](https://www.youtube.com/watch?v=4nxStxC1BJM) -- Detailed [guide on pricing models](https://docs.apify.com/platform/actors/running/actors-in-store) for Actors in Store +- Detailed [guide on pricing models](https://docs.apify.com/platform/using-actors/running/actors-in-store) for Actors in Store diff --git a/sources/academy/platform/getting_started/creating_actors.md b/sources/academy/platform/getting_started/creating_actors.md index 0e6e8d1eee..28dfb67dff 100644 --- a/sources/academy/platform/getting_started/creating_actors.md +++ b/sources/academy/platform/getting_started/creating_actors.md @@ -54,7 +54,7 @@ You will end up on a template detail page where you can see all the important in ### Using the template in the Web IDE {#web-ide} -By clicking **Use this template** button you will create the Actor in Apify Console and you will be moved to the **Code** tab with the [Web IDE](/platform/actors/development/quick-start/web-ide) where you can see the code of the template and start editing it. +By clicking **Use this template** button you will create the Actor in Apify Console and you will be moved to the **Code** tab with the [Web IDE](/platform/getting-started/web-ide) where you can see the code of the template and start editing it. > The Web IDE is a great tool for developing your Actor directly in Apify Console without the need to install or use any other software. @@ -133,13 +133,13 @@ The Actor takes the `url` from the input and then: 3. Extracts headings (H1 - H6) from the page. 4. Stores the extracted data. -The extracted data is stored in the [Dataset](/platform/storage/dataset) where you can preview it and download it. We'll show how to do that later in [Run the Actor](#run-the-actor) section. +The extracted data is stored in the [Dataset](/platform/core-concepts/storage/dataset) where you can preview it and download it. We'll show how to do that later in [Run the Actor](#run-the-actor) section. > Feel free to play around with the code and add some more features to it. For example, you can extract all the links from the page or extract all the images or completely change the logic of this template. Keep in mind that this template uses [input schema](/academy/deploying-your-code/input-schema) defined in the `.actor/input_schema.json` file and linked to the `.actor/actor.json`. If you want to change the input schema, you need to change it in those files as well. Learn more about the Actor input and output [in the next page](/academy/getting-started/inputs-outputs). ## Build the Actor 🧱 {#build-an-actor} -In order to run the Actor, you need to [build](/platform/actors/development/builds-and-runs/builds) it first. Click on the **Build** button at the bottom of the page or **Build now** button right under the code editor. +In order to run the Actor, you need to [build](/platform/using-actors/development/builds-and-runs/builds) it first. Click on the **Build** button at the bottom of the page or **Build now** button right under the code editor. ![Build the Actor](./images/build-actor.png) diff --git a/sources/academy/platform/getting_started/inputs_outputs.md b/sources/academy/platform/getting_started/inputs_outputs.md index 564e31f4d8..be48d53dc1 100644 --- a/sources/academy/platform/getting_started/inputs_outputs.md +++ b/sources/academy/platform/getting_started/inputs_outputs.md @@ -65,7 +65,7 @@ Then, replace everything in **INPUT_SCHEMA.json** with this: } ``` -> If you're interested in learning more about how the code works, and what the **INPUT_SCHEMA.json** means, read about [inputs](/sdk/js/docs/examples/accept-user-input) and [adding data to a dataset](/sdk/js/docs/examples/add-data-to-dataset) in the Apify SDK documentation, and refer to the [input schema docs](/platform/actors/development/actor-definition/input-schema/specification/v1#integer). +> If you're interested in learning more about how the code works, and what the **INPUT_SCHEMA.json** means, read about [inputs](/sdk/js/docs/examples/accept-user-input) and [adding data to a dataset](/sdk/js/docs/examples/add-data-to-dataset) in the Apify SDK documentation, and refer to the [input schema docs](/platform/using-actors/development/actor-definition/input-schema/specification/v1#integer). Finally, **Save** and **Build** the Actor just as you did in the previous lesson. @@ -89,7 +89,7 @@ On the results tab, there are a whole lot of options for which format to view/do There's our solution! Did it work for you as well? Now, we can download the data right from the results tab to be used elsewhere, or even programmatically retrieve it by using [Apify's API](/api/v2) (we'll be discussing how to do this in the next lesson). -It's important to note that the default dataset of the Actor, which we pushed our solution to, will be retained for 7 days. If we wanted the data to be retained for an indefinite period of time, we'd have to use a named dataset. For more information about named storages vs unnamed storages, read a bit about [data retention on the Apify platform](/platform/storage/usage#data-retention). +It's important to note that the default dataset of the Actor, which we pushed our solution to, will be retained for 7 days. If we wanted the data to be retained for an indefinite period of time, we'd have to use a named dataset. For more information about named storages vs unnamed storages, read a bit about [data retention on the Apify platform](/platform/core-concepts/storage/usage#data-retention). ## Next up {#next} diff --git a/sources/academy/platform/running_a_web_server.md b/sources/academy/platform/running_a_web_server.md index 9c30d4d1a1..027b369936 100644 --- a/sources/academy/platform/running_a_web_server.md +++ b/sources/academy/platform/running_a_web_server.md @@ -21,7 +21,7 @@ Running a web server in an Actor is a piece of cake! Each Actor run is available If you start a web server on the port defined by the **APIFY_CONTAINER_PORT** environment variable (the default value is **4321**), the container URL becomes available and gets displayed in the **Live View** tab in the Actor run console. -For more details, see [the documentation](/platform/actors/development/programming-interface/container-web-server). +For more details, see [the documentation](/platform/using-actors/development/programming-interface/container-web-server). ## Building the Actor {#building-the-actor} diff --git a/sources/academy/tutorials/api/index.md b/sources/academy/tutorials/api/index.md index 8c1f212c93..596c0f3d8b 100644 --- a/sources/academy/tutorials/api/index.md +++ b/sources/academy/tutorials/api/index.md @@ -12,7 +12,7 @@ slug: /api --- -This section explains how you can run [Apify Actors](/platform/actors) using Apify's [API](/api/v2), retrieve their results, and integrate them into your own product and workflows. You can do this using a raw HTTP client, or you can benefit from using one of our API clients for: +This section explains how you can run [Apify Actors](/platform/using-actors) using Apify's [API](/api/v2), retrieve their results, and integrate them into your own product and workflows. You can do this using a raw HTTP client, or you can benefit from using one of our API clients for: - [JavaScript](/api/client/js/) - [Python](/api/client/python) diff --git a/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md b/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md index a11e82f121..1d1c2468fa 100644 --- a/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md +++ b/sources/academy/tutorials/api/run_actor_and_retrieve_data_via_api.md @@ -11,7 +11,7 @@ slug: /api/run-actor-and-retrieve-data-via-api import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -The most popular way of [integrating](https://help.apify.com/en/collections/1669769-integrations) the Apify platform with an external project/application is by programmatically running an [Actor](/platform/actors) or [task](/platform/actors/running/tasks), waiting for it to complete its run, then collecting its data and using it within the project. Follow this tutorial to have an idea on how to approach this, it isn't as complicated as it sounds! +The most popular way of [integrating](https://help.apify.com/en/collections/1669769-integrations) the Apify platform with an external project/application is by programmatically running an [Actor](/platform/using-actors) or [task](/platform/using-actors/running/tasks), waiting for it to complete its run, then collecting its data and using it within the project. Follow this tutorial to have an idea on how to approach this, it isn't as complicated as it sounds! > Remember to check out our [API documentation](/api/v2) with examples in different languages and a live API console. We also recommend testing the API with a desktop client like [Postman](https://www.postman.com/) or [Insomnia](https://insomnia.rest). @@ -25,7 +25,7 @@ If the Actor being run via API takes 5 minutes or less to complete a typical run ## Run an Actor or task {#run-an-actor-or-task} -> If you are unsure about the differences between an Actor and a task, you can read about them in the [tasks](/platform/actors/running/tasks) documentation. In brief, tasks are pre-configured inputs for Actors. +> If you are unsure about the differences between an Actor and a task, you can read about them in the [tasks](/platform/using-actors/running/tasks) documentation. In brief, tasks are pre-configured inputs for Actors. The API endpoints and usage (for both sync and async) for [Actors](/api/v2#tag/ActorsRun-collection/operation/act_runs_post) and [tasks](/api/v2/actor-task-runs-post) are essentially the same. @@ -61,7 +61,7 @@ We can also add settings for the Actor (which will override the default settings https://api.apify.com/v2/acts/ACTOR_NAME_OR_ID/runs?token=YOUR_TOKEN&memory=8192&build=beta ``` -This works in almost exactly the same way for both Actors and tasks; however, for tasks, there is no reason to specify a [`build`](/platform/actors/development/builds-and-runs/builds) parameter, as a task already has only one specific Actor build which cannot be changed with query parameters. +This works in almost exactly the same way for both Actors and tasks; however, for tasks, there is no reason to specify a [`build`](/platform/using-actors/development/builds-and-runs/builds) parameter, as a task already has only one specific Actor build which cannot be changed with query parameters. ### Input JSON {#input-json} @@ -149,7 +149,7 @@ If your synchronous run exceeds the 5-minute time limit, the response will be a ### Synchronous runs with dataset output {#synchronous-runs-with-dataset-output} -Most Actor runs will store their data in the default [dataset](/platform/storage/dataset). The Apify API provides **run-sync-get-dataset-items** endpoints for [Actors](/api/v2/act-run-sync-get-dataset-items-post) and [tasks](/api/v2/actor-task-run-sync-get-dataset-items-post), which allow you to run an Actor and receive the items from the default dataset once the run has finished. +Most Actor runs will store their data in the default [dataset](/platform/core-concepts/storage/dataset). The Apify API provides **run-sync-get-dataset-items** endpoints for [Actors](/api/v2/act-run-sync-get-dataset-items-post) and [tasks](/api/v2/actor-task-run-sync-get-dataset-items-post), which allow you to run an Actor and receive the items from the default dataset once the run has finished. Here is a Node.js example of calling a task via the API and logging the dataset items to the console: @@ -186,7 +186,7 @@ items.forEach((item) => { ### Synchronous runs with key-value store output {#synchronous-runs-with-key-value-store-output} -[Key-value stores](/platform/storage/key-value-store) are useful for storing files like images, HTML snapshots, or JSON data. The Apify API provides **run-sync** endpoints for [Actors](/api/v2/act-run-sync-post) and [tasks](/api/v2/actor-task-run-sync-post), which allow you to run a specific task and receive the output. By default, they return the `OUTPUT` record from the default key-value store. +[Key-value stores](/platform/core-concepts/storage/key-value-store) are useful for storing files like images, HTML snapshots, or JSON data. The Apify API provides **run-sync** endpoints for [Actors](/api/v2/act-run-sync-post) and [tasks](/api/v2/actor-task-run-sync-post), which allow you to run a specific task and receive the output. By default, they return the `OUTPUT` record from the default key-value store. ## Asynchronous flow {#asynchronous-flow} @@ -248,11 +248,11 @@ Once a status of `SUCCEEDED` or `FAILED` has been received, we know the run has Unless you used the [synchronous call](#synchronous-flow) mentioned above, you will have to make one additional request to the API to retrieve the data. -The **run info** JSON also contains the IDs of the default [dataset](/platform/storage/dataset) and [key-value store](/platform/storage/key-value-store) that are allocated separately for each run, which is usually everything you need. The fields are called `defaultDatasetId` and `defaultKeyValueStoreId`. +The **run info** JSON also contains the IDs of the default [dataset](/platform/core-concepts/storage/dataset) and [key-value store](/platform/core-concepts/storage/key-value-store) that are allocated separately for each run, which is usually everything you need. The fields are called `defaultDatasetId` and `defaultKeyValueStoreId`. #### Retrieving a dataset {#retrieve-a-dataset} -> If you are scraping products, or any list of items with similar fields, the [dataset](/platform/storage/dataset) should be your storage of choice. Don't forget though, that dataset items are immutable. This means that you can only add to the dataset, and not change the content that is already inside it. +> If you are scraping products, or any list of items with similar fields, the [dataset](/platform/core-concepts/storage/dataset) should be your storage of choice. Don't forget though, that dataset items are immutable. This means that you can only add to the dataset, and not change the content that is already inside it. To retrieve the data from a dataset, send a GET request to the [**Get items**](/api/v2/dataset-items-get) endpoint and pass the `defaultDatasetId` into the URL. For a GET request to the default dataset, no token is needed. @@ -272,7 +272,7 @@ https://api.apify.com/v2/datasets/DATASET_ID/items?format=csv&offset=250000 #### Retrieving a key-value store {#retrieve-a-key-value-store} -> [Key-value stores](/platform/storage/key-value-store) are mainly useful if you have a single output or any kind of files that cannot be [stringified](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify) (such as images or PDFs). +> [Key-value stores](/platform/core-concepts/storage/key-value-store) are mainly useful if you have a single output or any kind of files that cannot be [stringified](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify) (such as images or PDFs). When you want to retrieve something from a key-value store, the `defaultKeyValueStoreId` is _not_ enough. You also need to know the name (or **key**) of the record you want to retrieve. diff --git a/sources/academy/tutorials/api/using_apify_from_php.md b/sources/academy/tutorials/api/using_apify_from_php.md index 3a1575d1fb..1d298f1c45 100644 --- a/sources/academy/tutorials/api/using_apify_from_php.md +++ b/sources/academy/tutorials/api/using_apify_from_php.md @@ -81,7 +81,7 @@ $data = $parsedResponse['data']; echo \json_encode($data, JSON_PRETTY_PRINT); ``` -You should see information about the run, including its ID and the ID of its default [dataset](/platform/storage/dataset). Take note of these, we will need them later. +You should see information about the run, including its ID and the ID of its default [dataset](/platform/core-concepts/storage/dataset). Take note of these, we will need them later. ## Getting the results from dataset @@ -127,7 +127,7 @@ All the available parameters are described in [our API reference](/api/v2/datase ## Getting the results from key-value stores -Datasets are great for structured data, but are not suited for binary files like images or PDFs. In these cases, Actors store their output in [key-value stores](/platform/storage/key-value-store). One such Actor is the **HTML String To PDF** ([mhamas/html-string-to-pdf](https://apify.com/mhamas/html-string-to-pdf)) converter. Let's run it. +Datasets are great for structured data, but are not suited for binary files like images or PDFs. In these cases, Actors store their output in [key-value stores](/platform/core-concepts/storage/key-value-store). One such Actor is the **HTML String To PDF** ([mhamas/html-string-to-pdf](https://apify.com/mhamas/html-string-to-pdf)) converter. Let's run it. ```php $response = $client->post('acts/mhamas~html-string-to-pdf/runs', [ @@ -229,7 +229,7 @@ $response = $client->post('acts/mhamas~html-string-to-pdf/runs', [ ## How to use Apify Proxy -Let's use another important feature: [proxy](/platform/proxy). If you want to make sure that your server's IP address won't get blocked somewhere when making requests, you can use the automatic proxy selection mode. +Let's use another important feature: [proxy](/platform/core-concepts/proxy). If you want to make sure that your server's IP address won't get blocked somewhere when making requests, you can use the automatic proxy selection mode. ```php $client = new \GuzzleHttp\Client([ @@ -260,7 +260,7 @@ $response = $client->get("https://api.apify.com/v2/browser-info"); echo $response->getBody(); ``` -[See the proxy docs](/platform/proxy/usage) for more details on using specific proxies. +[See the proxy docs](/platform/core-concepts/proxy/usage) for more details on using specific proxies. ## Feedback diff --git a/sources/academy/tutorials/apify_scrapers/cheerio_scraper.md b/sources/academy/tutorials/apify_scrapers/cheerio_scraper.md index 586a835b40..9ae834398b 100644 --- a/sources/academy/tutorials/apify_scrapers/cheerio_scraper.md +++ b/sources/academy/tutorials/apify_scrapers/cheerio_scraper.md @@ -502,7 +502,7 @@ Thank you for reading this whole tutorial! Really! It's important to us that our ## What's next * Check out the [Apify SDK](https://docs.apify.com/sdk) and its [Getting started](https://docs.apify.com/sdk/js/docs/guides/apify-platform) tutorial if you'd like to try building your own Actors. It's a bit more complex and involved than writing a `pageFunction`, but it allows you to fine-tune all the details of your scraper to your liking. -* [Take a deep dive into Actors](/platform/actors), from how they work to [publishing](/platform/actors/publishing) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. +* [Take a deep dive into Actors](/platform/using-actors), from how they work to [publishing](/platform/publishing-and-monetization/publish) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. * Found out you're not into the coding part but would still to use Apify Actors? Check out our [ready-made solutions](https://apify.com/store) or [order a custom Actor](https://apify.com/contact-sales) from an Apify-certified developer. diff --git a/sources/academy/tutorials/apify_scrapers/getting_started.md b/sources/academy/tutorials/apify_scrapers/getting_started.md index 9b05130eba..18fdbb5d2e 100644 --- a/sources/academy/tutorials/apify_scrapers/getting_started.md +++ b/sources/academy/tutorials/apify_scrapers/getting_started.md @@ -17,7 +17,7 @@ Welcome to the getting started tutorial! It will walk you through creating your It doesn't matter whether you arrived here from **Web Scraper** ([apify/web-scraper](https://apify.com/apify/web-scraper)), **Puppeteer Scraper** ([apify/puppeteer-scraper](https://apify.com/apify/puppeteer-scraper)) or **Cheerio Scraper** ([apify/cheerio-scraper](https://apify.com/apify/cheerio-scraper)). All of them are **Actors** and for now, let's think of an **Actor** as an application that you can use with your own configuration. **apify/web-scraper** is therefore an application called **web-scraper**, built by **apify**, that you can configure to scrape any webpage. We call these configurations **tasks**. -> If you need help choosing the right scraper, see this [great article](https://help.apify.com/en/articles/3024655-choosing-the-right-solution). If you want to learn more about Actors in general, you can read our [Actors page](https://apify.com/actors) or [browse the documentation](/platform/actors). +> If you need help choosing the right scraper, see this [great article](https://help.apify.com/en/articles/3024655-choosing-the-right-solution). If you want to learn more about Actors in general, you can read our [Actors page](https://apify.com/actors) or [browse the documentation](/platform/using-actors). You can create 10 different **tasks** for 10 different websites, with very different options, but there will always be just one **Actor**, the `apify/*-scraper` you chose. This is the essence of tasks. They are nothing but **saved configurations** of the Actor that you can run repeatedly. @@ -35,7 +35,7 @@ This takes you to the **Input and options** tab of the task configuration. Befor Scroll down to the **Performance and limits** section and set the **Max pages per run** option to **10**. This tells your task to finish after 10 pages have been visited. We don't need to crawl the whole domain to see that the Actor works. -> This also helps with keeping your [compute unit](/platform/actors/running/usage-and-resources) (CU) consumption low. To get an idea, our free plan includes 10 CUs and this run will consume about 0.04 CU, so you can run it 250 times a month for free. If you accidentally go over the limit, no worries, we won't charge you for it. You just won't be able to run more tasks that month. +> This also helps with keeping your [compute unit](/platform/using-actors/running/usage-and-resources) (CU) consumption low. To get an idea, our free plan includes 10 CUs and this run will consume about 0.04 CU, so you can run it 250 times a month for free. If you accidentally go over the limit, no worries, we won't charge you for it. You just won't be able to run more tasks that month. Now click **Save & Run**! *(in the bottom-left part of your screen)* @@ -254,7 +254,7 @@ When a **Pseudo URL** is set, the scraper attempts to enqueue matching links on The `pageFunction` may only return nothing, `null`, `Object` or `Object[]`. If an `Object` is returned, it will be saved as a single result. Returning an `Array` of `Objects` will save each item in the array as a result. -The scraping results are saved in a [dataset](/platform/storage/dataset) (one of the tabs in the run console, as you may remember). It behaves like a table. Each item is a row in the table and its properties are its columns. Returning the following `Object`: +The scraping results are saved in a [dataset](/platform/core-concepts/storage/dataset) (one of the tabs in the run console, as you may remember). It behaves like a table. Each item is a row in the table and its properties are its columns. Returning the following `Object`: ```js async function pageFunction(context) { diff --git a/sources/academy/tutorials/apify_scrapers/puppeteer_scraper.md b/sources/academy/tutorials/apify_scrapers/puppeteer_scraper.md index 130713691a..8a918facff 100644 --- a/sources/academy/tutorials/apify_scrapers/puppeteer_scraper.md +++ b/sources/academy/tutorials/apify_scrapers/puppeteer_scraper.md @@ -822,7 +822,7 @@ Thank you for reading this whole tutorial! Really! It's important to us that our ## What's next - Check out the [Apify SDK](https://docs.apify.com/sdk) and its [Getting started](https://docs.apify.com/sdk/js/docs/guides/apify-platform) tutorial if you'd like to try building your own Actors. It's a bit more complex and involved than writing a `pageFunction`, but it allows you to fine-tune all the details of your scraper to your liking. -- [Take a deep dive into Actors](/platform/actors), from how they work to [publishing](/platform/actors/publishing) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. +- [Take a deep dive into Actors](/platform/using-actors), from how they work to [publishing](/platform/publishing-and-monetization/publish) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. - Found out you're not into the coding part but would still to use Apify Actors? Check out our [ready-made solutions](https://apify.com/store) or [order a custom Actor](https://apify.com/contact-sales) from an Apify-certified developer. diff --git a/sources/academy/tutorials/apify_scrapers/web_scraper.md b/sources/academy/tutorials/apify_scrapers/web_scraper.md index 3b468e198f..1303b935b2 100644 --- a/sources/academy/tutorials/apify_scrapers/web_scraper.md +++ b/sources/academy/tutorials/apify_scrapers/web_scraper.md @@ -556,7 +556,7 @@ Thank you for reading this whole tutorial! Really! It's important to us that our ## What's next - Check out the [Apify SDK](https://docs.apify.com/sdk) and its [Getting started](https://docs.apify.com/sdk/js/docs/guides/apify-platform) tutorial if you'd like to try building your own Actors. It's a bit more complex and involved than writing a `pageFunction`, but it allows you to fine-tune all the details of your scraper to your liking. -- [Take a deep dive into Actors](/platform/actors), from how they work to [publishing](/platform/actors/publishing) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. +- [Take a deep dive into Actors](/platform/using-actors), from how they work to [publishing](/platform/publishing-and-monetization/publish) them in Apify Store, and even [making money](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) on Actors. - Found out you're not into the coding part but would still to use Apify Actors? Check out our [ready-made solutions](https://apify.com/store) or [order a custom Actor](https://apify.com/contact-sales) from an Apify-certified developer. diff --git a/sources/academy/tutorials/node_js/analyzing_pages_and_fixing_errors.md b/sources/academy/tutorials/node_js/analyzing_pages_and_fixing_errors.md index 892a3dd59b..c9b32686bf 100644 --- a/sources/academy/tutorials/node_js/analyzing_pages_and_fixing_errors.md +++ b/sources/academy/tutorials/node_js/analyzing_pages_and_fixing_errors.md @@ -127,7 +127,7 @@ Logging and snapshotting are great tools but once you reach a certain run size, ## With the Apify SDK {#with-the-apify-sdk} -This example extends our snapshot solution above by creating a [named dataset](/platform/storage/usage#named-and-unnamed-storages) (named datasets have infinite retention), where we will accumulate error reports. Those reports will explain what happened and will link to a saved snapshot, so we can do a quick visual check. +This example extends our snapshot solution above by creating a [named dataset](/platform/core-concepts/storage/usage#named-and-unnamed-storages) (named datasets have infinite retention), where we will accumulate error reports. Those reports will explain what happened and will link to a saved snapshot, so we can do a quick visual check. ```js import { Actor } from 'apify'; diff --git a/sources/academy/tutorials/node_js/apify_free_google_serp_api.md b/sources/academy/tutorials/node_js/apify_free_google_serp_api.md index ac643b365b..ad475aa052 100644 --- a/sources/academy/tutorials/node_js/apify_free_google_serp_api.md +++ b/sources/academy/tutorials/node_js/apify_free_google_serp_api.md @@ -10,7 +10,7 @@ You need to regularly grab SERP data about your target keywords? Apify provides ![Apify Google SERP API](./images/gserp-api.png) -Hit `Save & Run` and you'll have the downloaded data as soon as the query finishes. To have it run at a regular frequency, you can set up the task to run on an [automatic schedule](/platform/schedules#setting-up-a-new-schedule). +Hit `Save & Run` and you'll have the downloaded data as soon as the query finishes. To have it run at a regular frequency, you can set up the task to run on an [automatic schedule](/platform/using-actors/schedules#setting-up-a-new-schedule). To run from the API, send a [synchronous POST request](/api/v2/actor-task-run-sync-get-dataset-items-post) to an endpoint such as `https://api.apify.com/v2/acts/TASK_NAME_OR_ID/runs?token=YOUR_TOKEN`. Include any required input in a JSON object in the request's body. diff --git a/sources/academy/tutorials/node_js/caching_responses_in_puppeteer.md b/sources/academy/tutorials/node_js/caching_responses_in_puppeteer.md index 70b0f2ab07..b0fd4b47b2 100644 --- a/sources/academy/tutorials/node_js/caching_responses_in_puppeteer.md +++ b/sources/academy/tutorials/node_js/caching_responses_in_puppeteer.md @@ -99,7 +99,7 @@ After implementing this code, we can run the scraper again. ![Good run results](./images/good-run-results.png) -Looking at the statistics, caching responses in Puppeteer brought the traffic down from 177MB to 13.4MB, which is a reduction of data transfer by 92%. The related screenshots can be found [here](https://my.apify.com/storage/key-value/iWQ3mQE2XsLA2eErL). +Looking at the statistics, caching responses in Puppeteer brought the traffic down from 177MB to 13.4MB, which is a reduction of data transfer by 92%. [View the related screenshots](https://my.apify.com/storage/key-value/iWQ3mQE2XsLA2eErL) in the key-value store. It did not speed up the crawler, but that is only because the crawler is set to wait until the network is nearly idle, and CNN has a lot of tracking and analytics scripts that keep the network busy. diff --git a/sources/academy/tutorials/node_js/filter_blocked_requests_using_sessions.md b/sources/academy/tutorials/node_js/filter_blocked_requests_using_sessions.md index bf0fc4b30a..44ecc5b327 100644 --- a/sources/academy/tutorials/node_js/filter_blocked_requests_using_sessions.md +++ b/sources/academy/tutorials/node_js/filter_blocked_requests_using_sessions.md @@ -21,13 +21,13 @@ You want to crawl a website with a proxy pool, but most of your proxies are bloc 5. The proxies actually got banned before anyone used them to crawl the website because they use anti-bot protection that bans proxies across websites (e.g. Cloudflare). -Nobody can make sure that a proxy will work infinitely. The only real solution to this problem is to use [residential proxies](/platform/proxy/residential-proxy), but they can sometimes be too costly. +Nobody can make sure that a proxy will work infinitely. The only real solution to this problem is to use [residential proxies](/platform/core-concepts/proxy/residential-proxy), but they can sometimes be too costly. However, usually, at least some of our proxies work. To crawl successfully, it is therefore imperative to handle blocked requests properly. You first need to discover that you are blocked, which usually means that either your request returned status greater or equal to 400 (it didn't return the proper response) or that the page displayed a captcha. To ensure that this bad request is retried, you usually throw an error and it gets automatically retried later (our [SDK](/sdk/js/) handles this for you). Check out [this article](https://docs.apify.com/academy/node-js/handle-blocked-requests-puppeteer) as inspiration for how to handle this situation with `PuppeteerCrawler` class. ### Solution -Now we are able to retry bad requests and eventually unless all of our proxies get banned, we should be able to successfully crawl what we want. The problem is that it takes too long and our log is full of errors. Fortunately, we can overcome this with [proxy sessions](/platform/proxy/datacenter-proxy#username-parameters) (look at the proxy and SDK documentation for how to use them in your Actors.) +Now we are able to retry bad requests and eventually unless all of our proxies get banned, we should be able to successfully crawl what we want. The problem is that it takes too long and our log is full of errors. Fortunately, we can overcome this with [proxy sessions](/platform/core-concepts/proxy/datacenter-proxy#username-parameters) (look at the proxy and SDK documentation for how to use them in your Actors.) First we define `sessions`  object at the top of our code (in global scope) to hold the state of our working sessions. @@ -180,7 +180,7 @@ const gotoFunction = async ({ request, page }) => { }; ``` -Now we have access to the session in the `handlePageFunction` and the rest of the logic is the same as in the first example. We extract the session from the userData, try/catch the whole code and on success we add the session and on error we delete it. Also it is useful to retire the browser completely (check [here](https://docs.apify.com/academy/node-js/handle-blocked-requests-puppeteer) for reference) since the other requests will probably have similar problem. +Now we have access to the session in the `handlePageFunction` and the rest of the logic is the same as in the first example. We extract the session from the userData, try/catch the whole code and on success we add the session and on error we delete it. Also it is useful to retire the browser completely (see [handling blocked requests with Puppeteer](https://docs.apify.com/academy/node-js/handle-blocked-requests-puppeteer) for reference) since the other requests will probably have similar problem. ```js const handlePageFunction = async ({ request, page, puppeteerPool }) => { diff --git a/sources/academy/tutorials/node_js/handle_blocked_requests_puppeteer.md b/sources/academy/tutorials/node_js/handle_blocked_requests_puppeteer.md index 313802e0be..55ef2f2d27 100644 --- a/sources/academy/tutorials/node_js/handle_blocked_requests_puppeteer.md +++ b/sources/academy/tutorials/node_js/handle_blocked_requests_puppeteer.md @@ -5,7 +5,7 @@ sidebar_position: 15.9 slug: /node-js/handle-blocked-requests-puppeteer --- -One of the main defense mechanisms websites use to ensure they are not scraped by bots is allowing only a limited number of requests from a specific IP address. That's why Apify provides a [proxy](https://docs.apify.com/platform/proxy) component with intelligent rotation. With a large enough pool of proxies, you can multiply the number of allowed requests per day to cover your crawling needs. Let's look at how we can rotate proxies when using our [JavaScript SDK](https://github.com/apify/apify-sdk-js). +One of the main defense mechanisms websites use to ensure they are not scraped by bots is allowing only a limited number of requests from a specific IP address. That's why Apify provides a [proxy](https://docs.apify.com/platform/core-concepts/proxy) component with intelligent rotation. With a large enough pool of proxies, you can multiply the number of allowed requests per day to cover your crawling needs. Let's look at how we can rotate proxies when using our [JavaScript SDK](https://github.com/apify/apify-sdk-js). # BasicCrawler diff --git a/sources/academy/tutorials/node_js/scraping_urls_list_from_google_sheets.md b/sources/academy/tutorials/node_js/scraping_urls_list_from_google_sheets.md index fa0c546a7d..f76372a8c5 100644 --- a/sources/academy/tutorials/node_js/scraping_urls_list_from_google_sheets.md +++ b/sources/academy/tutorials/node_js/scraping_urls_list_from_google_sheets.md @@ -5,7 +5,7 @@ sidebar_position: 15 slug: /node-js/scraping-urls-list-from-google-sheets --- -You can export URLs from [Google Sheets](https://workspace.google.com/products/sheets/) such as [this one](https://docs.google.com/spreadsheets/d/1-2mUcRAiBbCTVA5KcpFdEYWflLMLp9DDU3iJutvES4w) directly into an [Actor](/platform/actors)'s Start URLs field. +You can export URLs from [Google Sheets](https://workspace.google.com/products/sheets/) such as [this one](https://docs.google.com/spreadsheets/d/1-2mUcRAiBbCTVA5KcpFdEYWflLMLp9DDU3iJutvES4w) directly into an [Actor](/platform/using-actors)'s Start URLs field. 1. Make sure the spreadsheet has one sheet and a simple structure to help the Actor find the URLs. diff --git a/sources/academy/tutorials/python/process_data_using_python.md b/sources/academy/tutorials/python/process_data_using_python.md index 5e72eaddb9..e194a8d346 100644 --- a/sources/academy/tutorials/python/process_data_using_python.md +++ b/sources/academy/tutorials/python/process_data_using_python.md @@ -19,7 +19,7 @@ In this tutorial, we will use the Actor we created in the [previous tutorial](/a ## Processing previously scraped data -In the previous tutorial, we set out to select our next holiday destination based on the forecast of the upcoming weather there. We have written an Actor that scrapes the BBC Weather forecast for the upcoming two weeks for three destinations: Prague, New York, and Honolulu. It then saves the scraped data to a [dataset](/platform/storage/dataset) on the Apify platform. +In the previous tutorial, we set out to select our next holiday destination based on the forecast of the upcoming weather there. We have written an Actor that scrapes the BBC Weather forecast for the upcoming two weeks for three destinations: Prague, New York, and Honolulu. It then saves the scraped data to a [dataset](/platform/core-concepts/storage/dataset) on the Apify platform. Now, we need to process the scraped data and make a visualization that will help us decide which location has the best weather, and will therefore become our next holiday destination. @@ -110,7 +110,7 @@ axes.legend(loc='best') axes.figure.tight_layout() ``` -As the last step, we need to save the plot to a record in a [key-value store](/platform/storage/key-value-store) on the Apify platform, so that we can access it later. We save the rendered figure with the plot to an in-memory buffer, and then save the contents of that buffer to the default key-value store of the Actor run through its resource subclient. +As the last step, we need to save the plot to a record in a [key-value store](/platform/core-concepts/storage/key-value-store) on the Apify platform, so that we can access it later. We save the rendered figure with the plot to an in-memory buffer, and then save the contents of that buffer to the default key-value store of the Actor run through its resource subclient. ```py # Get the resource sub-client for working with the default key-value store of the run diff --git a/sources/academy/tutorials/python/scrape_data_python.md b/sources/academy/tutorials/python/scrape_data_python.md index df8dfcdbfd..22eba2e35d 100644 --- a/sources/academy/tutorials/python/scrape_data_python.md +++ b/sources/academy/tutorials/python/scrape_data_python.md @@ -219,7 +219,7 @@ Earlier in this tutorial, we learned how to scrape data from the web in Python u ## Processing previously scraped data -In the previous tutorial, we set out to select our next holiday destination based on the forecast of the upcoming weather there. We have written an Actor that scrapes the BBC Weather forecast for the upcoming two weeks for three destinations: Prague, New York, and Honolulu. It then saves the scraped data to a [dataset](/platform/storage/dataset) on the Apify platform. +In the previous tutorial, we set out to select our next holiday destination based on the forecast of the upcoming weather there. We have written an Actor that scrapes the BBC Weather forecast for the upcoming two weeks for three destinations: Prague, New York, and Honolulu. It then saves the scraped data to a [dataset](/platform/core-concepts/storage/dataset) on the Apify platform. Now, we need to process the scraped data and make a visualization that will help us decide which location has the best weather, and will therefore become our next holiday destination. @@ -310,7 +310,7 @@ axes.legend(loc='best') axes.figure.tight_layout() ``` -As the last step, we need to save the plot to a record in a [key-value store](/platform/storage/key-value-store) on the Apify platform, so that we can access it later. We save the rendered figure with the plot to an in-memory buffer, and then save the contents of that buffer to the default key-value store of the Actor run through its resource subclient. +As the last step, we need to save the plot to a record in a [key-value store](/platform/core-concepts/storage/key-value-store) on the Apify platform, so that we can access it later. We save the rendered figure with the plot to an in-memory buffer, and then save the contents of that buffer to the default key-value store of the Actor run through its resource subclient. ```py # Get the resource sub-client for working with the default key-value store of the run diff --git a/sources/academy/webscraping/advanced_web_scraping/crawling/crawling-with-search.md b/sources/academy/webscraping/advanced_web_scraping/crawling/crawling-with-search.md index 1388b31329..c782f4efe3 100644 --- a/sources/academy/webscraping/advanced_web_scraping/crawling/crawling-with-search.md +++ b/sources/academy/webscraping/advanced_web_scraping/crawling/crawling-with-search.md @@ -75,7 +75,7 @@ Some sites will allow you to construct non-overlapping ranges. For example, you Non-overlapping ranges should remove the possibility of duplicate products (unless a [listing has multiple values](#can-a-listing-have-more-values)) and the lowest number of pages. -If the website supports only overlapping ranges (e.g. **$0-$5**, **$5–10**), it is not a big problem. Only a small portion of the listings will be duplicates, and they can be removed using a [request queue](/platform/storage/request-queue). +If the website supports only overlapping ranges (e.g. **$0-$5**, **$5–10**), it is not a big problem. Only a small portion of the listings will be duplicates, and they can be removed using a [request queue](/platform/core-concepts/storage/request-queue). #### Can a listing have more values? {#can-a-listing-have-more-values} diff --git a/sources/academy/webscraping/anti_scraping/index.md b/sources/academy/webscraping/anti_scraping/index.md index 5cb7f63466..3ac6dcbe09 100644 --- a/sources/academy/webscraping/anti_scraping/index.md +++ b/sources/academy/webscraping/anti_scraping/index.md @@ -22,7 +22,7 @@ In development, it is crucial to check and adjust the configurations related to If you don't have time to read about the theory behind anti-scraping protections to fine-tune your scraping project and instead you need to get unblocked ASAP, here are some quick tips: -- Use high-quality proxies. [Residential proxies](/platform/proxy/residential-proxy) are the least blocked. You can find many providers out there like Apify, BrightData, Oxylabs, NetNut, etc. +- Use high-quality proxies. [Residential proxies](/platform/core-concepts/proxy/residential-proxy) are the least blocked. You can find many providers out there like Apify, BrightData, Oxylabs, NetNut, etc. - Set **real-user-like HTTP settings** and **browser fingerprints**. [Crawlee](https://crawlee.dev/) uses statistically generated realistic HTTP headers and browser fingerprints by default for all of its crawlers. - Use a browser to pass bot capture challenges. We recommend [Playwright with Firefox](https://crawlee.dev/docs/examples/playwright-crawler-firefox) because it is not that common for scraping. You can also play with [non-headless mode](https://crawlee.dev/api/playwright-crawler/interface/PlaywrightCrawlerOptions#headless) and adjust other [fingerprint settings](https://crawlee.dev/api/browser-pool/interface/FingerprintGeneratorOptions). - Consider extracting data from **[private APIs](../api_scraping/index.md)** or **mobile app APIs**. They are usually much less protected. @@ -113,7 +113,7 @@ Because we here at Apify scrape for a living, we have discovered many popular an This is the most straightforward and standard protection, which is mainly implemented to prevent DDoS attacks, but it also works for blocking scrapers. Websites using rate limiting don't allow to more than some defined number of requests from one IP address in a certain time span. If the max-request number is low, then there is a high potential for false-positive due to IP address uniqueness, such as in large companies where hundreds of employees can share the same IP address. -> Learn more about rate limiting [here](./techniques/rate_limiting.md) +> Learn more about [rate limiting techniques](./techniques/rate_limiting.md) ### Header checking diff --git a/sources/academy/webscraping/scraping_basics_javascript2/13_platform.md b/sources/academy/webscraping/scraping_basics_javascript2/13_platform.md index cc1fc3b7d1..e0c45040d9 100644 --- a/sources/academy/webscraping/scraping_basics_javascript2/13_platform.md +++ b/sources/academy/webscraping/scraping_basics_javascript2/13_platform.md @@ -61,7 +61,7 @@ Success: You are logged in to Apify as user1234! ## Turning our program to an Actor -Every program that runs on the Apify platform first needs to be packaged as a so-called [Actor](https://docs.apify.com/platform/actors)—a standardized container with designated places for input and output. +Every program that runs on the Apify platform first needs to be packaged as a so-called [Actor](https://docs.apify.com/platform/using-actors)—a standardized container with designated places for input and output. Many [Actor templates](https://apify.com/templates/categories/javascript) simplify the setup for new projects. We'll skip those, as we're about to package an existing program. @@ -180,7 +180,7 @@ When the run finishes, the interface will turn green. On the **Output** tab, we :::info Accessing data -We don't need to click buttons to download the data. It's possible to retrieve it also using Apify's API, the `apify datasets` CLI command, or the JavaScript SDK. Learn more in the [Dataset docs](https://docs.apify.com/platform/storage/dataset). +We don't need to click buttons to download the data. It's possible to retrieve it also using Apify's API, the `apify datasets` CLI command, or the JavaScript SDK. Learn more in the [Dataset docs](https://docs.apify.com/platform/core-concepts/storage/dataset). ::: @@ -194,9 +194,9 @@ From now on, the Actor will execute daily. We can inspect each run, view logs, c ## Adding support for proxies -If monitoring shows that our scraper frequently fails to reach the Warehouse Shop website, it's likely being blocked. To avoid this, we can [configure proxies](https://docs.apify.com/platform/proxy) so our requests come from different locations, reducing the chances of detection and blocking. +If monitoring shows that our scraper frequently fails to reach the Warehouse Shop website, it's likely being blocked. To avoid this, we can [configure proxies](https://docs.apify.com/platform/core-concepts/proxy) so our requests come from different locations, reducing the chances of detection and blocking. -Proxy configuration is a type of [Actor input](https://docs.apify.com/platform/actors/running/input-and-output#input). Crawlee scrapers automatically connect their default dataset to the Actor output, but input must be handled manually. Inside the `.actor` directory we'll create a new file, `inputSchema.json`, with the following content: +Proxy configuration is a type of [Actor input](https://docs.apify.com/platform/using-actors/running/input-and-output#input). Crawlee scrapers automatically connect their default dataset to the Actor output, but input must be handled manually. Inside the `.actor` directory we'll create a new file, `inputSchema.json`, with the following content: ```json title=".actor/inputSchema.json" { @@ -328,4 +328,4 @@ We've reached the end of the course—congratulations! Together, we've built a p - Executes periodically without manual intervention, collecting data over time. - Uses proxies to avoid being blocked. -We hope this serves as a solid foundation for your next scraping project. Perhaps you'll even [start publishing scrapers](https://docs.apify.com/platform/actors/publishing) for others to use—for a fee? +We hope this serves as a solid foundation for your next scraping project. Perhaps you'll even [start publishing scrapers](https://docs.apify.com/platform/publishing-and-monetization/publish) for others to use—for a fee? diff --git a/sources/academy/webscraping/scraping_basics_python/13_platform.md b/sources/academy/webscraping/scraping_basics_python/13_platform.md index 23f042a048..01aa0eab3d 100644 --- a/sources/academy/webscraping/scraping_basics_python/13_platform.md +++ b/sources/academy/webscraping/scraping_basics_python/13_platform.md @@ -80,9 +80,9 @@ Info: To install additional Python packages, you need to activate the virtual en Inside the `warehouse-watchdog` directory, we should see a `src` subdirectory containing several Python files, including `main.py`. This is a sample Beautiful Soup scraper provided by the template. -The file contains a single asynchronous function, `main()`. At the beginning, it handles [input](https://docs.apify.com/platform/actors/running/input-and-output#input), then passes that input to a small crawler built on top of the Crawlee framework. +The file contains a single asynchronous function, `main()`. At the beginning, it handles [input](https://docs.apify.com/platform/using-actors/running/input-and-output#input), then passes that input to a small crawler built on top of the Crawlee framework. -Every program that runs on the Apify platform first needs to be packaged as a so-called [Actor](https://docs.apify.com/platform/actors)—a standardized container with designated places for input and output. Crawlee scrapers automatically connect their default dataset to the Actor output, but input must be handled explicitly in the code. +Every program that runs on the Apify platform first needs to be packaged as a so-called [Actor](https://docs.apify.com/platform/using-actors)—a standardized container with designated places for input and output. Crawlee scrapers automatically connect their default dataset to the Actor output, but input must be handled explicitly in the code. ![The expected file structure](./images/actor-file-structure.webp) @@ -266,7 +266,7 @@ When the run finishes, the interface will turn green. On the **Output** tab, we :::info Accessing data -We don't need to click buttons to download the data. It's possible to retrieve it also using Apify's API, the `apify datasets` CLI command, or the Python SDK. Learn more in the [Dataset docs](https://docs.apify.com/platform/storage/dataset). +We don't need to click buttons to download the data. It's possible to retrieve it also using Apify's API, the `apify datasets` CLI command, or the Python SDK. Learn more in the [Dataset docs](https://docs.apify.com/platform/core-concepts/storage/dataset). ::: @@ -280,7 +280,7 @@ From now on, the Actor will execute daily. We can inspect each run, view logs, c ## Adding support for proxies -If monitoring shows that our scraper frequently fails to reach the Warehouse Shop website, it's likely being blocked. To avoid this, we can [configure proxies](https://docs.apify.com/platform/proxy) so our requests come from different locations, reducing the chances of detection and blocking. +If monitoring shows that our scraper frequently fails to reach the Warehouse Shop website, it's likely being blocked. To avoid this, we can [configure proxies](https://docs.apify.com/platform/core-concepts/proxy) so our requests come from different locations, reducing the chances of detection and blocking. Proxy configuration is a type of Actor input, so let's start by reintroducing the necessary code. We'll update `warehouse-watchdog/src/main.py` like this: @@ -434,4 +434,4 @@ We've reached the end of the course—congratulations! Together, we've built a p - Executes periodically without manual intervention, collecting data over time. - Uses proxies to avoid being blocked. -We hope this serves as a solid foundation for your next scraping project. Perhaps you'll even [start publishing scrapers](https://docs.apify.com/platform/actors/publishing) for others to use—for a fee? +We hope this serves as a solid foundation for your next scraping project. Perhaps you'll even [start publishing scrapers](https://docs.apify.com/platform/publishing-and-monetization/publish) for others to use—for a fee? diff --git a/sources/platform/1-getting-started/_category_.yml b/sources/platform/1-getting-started/_category_.yml new file mode 100644 index 0000000000..956cec8526 --- /dev/null +++ b/sources/platform/1-getting-started/_category_.yml @@ -0,0 +1,3 @@ +label: 'Getting started' +position: 1 +collapsed: false \ No newline at end of file diff --git a/sources/platform/actors/development/quick_start/build_with_ai.md b/sources/platform/1-getting-started/build_with_ai.md similarity index 98% rename from sources/platform/actors/development/quick_start/build_with_ai.md rename to sources/platform/1-getting-started/build_with_ai.md index 904de4c8bc..863750c8f1 100644 --- a/sources/platform/actors/development/quick_start/build_with_ai.md +++ b/sources/platform/1-getting-started/build_with_ai.md @@ -1,8 +1,8 @@ --- title: Build with AI -sidebar_position: 3 +sidebar_position: 4 description: Learn how to set up your environment, choose the right tools, and establish workflows for effective vibe coding -slug: /actors/development/quick-start/build-with-ai +slug: /getting-started/build-with-ai toc_max_heading_level: 4 --- diff --git a/sources/platform/actors/development/quick_start/images/actor-build.png b/sources/platform/1-getting-started/images/actor-build.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-build.png rename to sources/platform/1-getting-started/images/actor-build.png diff --git a/sources/platform/actors/development/quick_start/images/actor-create-button.png b/sources/platform/1-getting-started/images/actor-create-button.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-create-button.png rename to sources/platform/1-getting-started/images/actor-create-button.png diff --git a/sources/platform/actors/development/quick_start/images/actor-create-templates.png b/sources/platform/1-getting-started/images/actor-create-templates.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-create-templates.png rename to sources/platform/1-getting-started/images/actor-create-templates.png diff --git a/sources/platform/actors/development/quick_start/images/actor-create.gif b/sources/platform/1-getting-started/images/actor-create.gif similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-create.gif rename to sources/platform/1-getting-started/images/actor-create.gif diff --git a/sources/platform/actors/development/quick_start/images/actor-input.png b/sources/platform/1-getting-started/images/actor-input.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-input.png rename to sources/platform/1-getting-started/images/actor-input.png diff --git a/sources/platform/actors/development/quick_start/images/actor-local-code.png b/sources/platform/1-getting-started/images/actor-local-code.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-local-code.png rename to sources/platform/1-getting-started/images/actor-local-code.png diff --git a/sources/platform/actors/development/quick_start/images/actor-local-run.png b/sources/platform/1-getting-started/images/actor-local-run.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-local-run.png rename to sources/platform/1-getting-started/images/actor-local-run.png diff --git a/sources/platform/actors/development/quick_start/images/actor-pull.png b/sources/platform/1-getting-started/images/actor-pull.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-pull.png rename to sources/platform/1-getting-started/images/actor-pull.png diff --git a/sources/platform/actors/development/quick_start/images/actor-run.png b/sources/platform/1-getting-started/images/actor-run.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-run.png rename to sources/platform/1-getting-started/images/actor-run.png diff --git a/sources/platform/actors/development/quick_start/images/actor-source-code.png b/sources/platform/1-getting-started/images/actor-source-code.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-source-code.png rename to sources/platform/1-getting-started/images/actor-source-code.png diff --git a/sources/platform/actors/development/quick_start/images/actor-templates.png b/sources/platform/1-getting-started/images/actor-templates.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/actor-templates.png rename to sources/platform/1-getting-started/images/actor-templates.png diff --git a/sources/platform/actors/development/quick_start/images/claude.png b/sources/platform/1-getting-started/images/claude.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/claude.png rename to sources/platform/1-getting-started/images/claude.png diff --git a/sources/platform/actors/development/quick_start/images/create-actor.png b/sources/platform/1-getting-started/images/create-actor.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/create-actor.png rename to sources/platform/1-getting-started/images/create-actor.png diff --git a/sources/platform/actors/development/quick_start/images/cursor.png b/sources/platform/1-getting-started/images/cursor.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/cursor.png rename to sources/platform/1-getting-started/images/cursor.png diff --git a/sources/platform/actors/development/quick_start/images/github-copilot.png b/sources/platform/1-getting-started/images/github-copilot.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/github-copilot.png rename to sources/platform/1-getting-started/images/github-copilot.png diff --git a/sources/platform/actors/development/quick_start/images/templates.png b/sources/platform/1-getting-started/images/templates.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/templates.png rename to sources/platform/1-getting-started/images/templates.png diff --git a/sources/platform/actors/development/quick_start/images/windsurf.png b/sources/platform/1-getting-started/images/windsurf.png similarity index 100% rename from sources/platform/actors/development/quick_start/images/windsurf.png rename to sources/platform/1-getting-started/images/windsurf.png diff --git a/sources/platform/actors/development/quick_start/index.mdx b/sources/platform/1-getting-started/index.mdx similarity index 77% rename from sources/platform/actors/development/quick_start/index.mdx rename to sources/platform/1-getting-started/index.mdx index 9063686d41..317c2420e3 100644 --- a/sources/platform/actors/development/quick_start/index.mdx +++ b/sources/platform/1-getting-started/index.mdx @@ -1,8 +1,8 @@ --- -title: Quick start +title: Getting started sidebar_position: 1 description: Create your first Actor using the Apify Web IDE or locally in your IDE. -slug: /actors/development/quick-start +slug: /getting-started --- import Card from "@site/src/components/Card"; @@ -14,13 +14,13 @@ import CardGrid from "@site/src/components/CardGrid"; :::info Before you build -Before you start building your own Actor, try out a couple of existing Actors from [Apify Store](https://apify.com/store). See the [Running Actors](../../running/index.md) section for more information on running existing Actors. +Before you start building your own Actor, try out a couple of existing Actors from [Apify Store](https://apify.com/store). See the [Running Actors](/platform/using-actors) section for more information on running existing Actors. ::: ## Technology stack -Any code that can run inside of a Docker container can be turned into Apify [Actor](../../index.mdx). This gives you freedom in choosing your technical stack, including programming language and technologies. +Any code that can run inside of a Docker container can be turned into Apify [Actor](/platform/using-actors). This gives you freedom in choosing your technical stack, including programming language and technologies. But to fully benefit from running on top of the Apify platform, we recommend you choose either JavaScript/Node.js or Python, where Apify provides first-level support regarding its SDK, API clients, and learning materials. @@ -36,12 +36,12 @@ You can develop Actor in two ways: Develop your Actor locally in your IDE and only deploy to the Apify platform when it is production ready. -This way, you benefit from your local setup for a better development and debugging experience. After you are done with the development, you can [deploy](./deployment) your Actor to the Apify platform. +This way, you benefit from your local setup for a better development and debugging experience. After you are done with the development, you can [deploy](/platform/building-actors) your Actor to the Apify platform. @@ -54,7 +54,7 @@ This is the fastest way to kick-start your Actor development and try out the Api @@ -65,6 +65,6 @@ Use the Apify toolset to build with AI: diff --git a/sources/platform/actors/development/quick_start/start_locally.md b/sources/platform/1-getting-started/start_locally.md similarity index 84% rename from sources/platform/actors/development/quick_start/start_locally.md rename to sources/platform/1-getting-started/start_locally.md index 9839e9c55b..6fed0a45ff 100644 --- a/sources/platform/actors/development/quick_start/start_locally.md +++ b/sources/platform/1-getting-started/start_locally.md @@ -1,8 +1,8 @@ --- title: Local development -sidebar_position: 1 +sidebar_position: 2 description: Create your first Actor locally on your machine, deploy it to the Apify platform, and run it in the cloud. -slug: /actors/development/quick-start/locally +slug: /getting-started/locally --- **Create your first Actor locally on your machine, deploy it to the Apify platform, and run it in the cloud.** @@ -71,7 +71,7 @@ You'll see output similar to this in your terminal: ```bash INFO System info {"apifyVersion":"3.4.3","apifyClientVersion":"2.12.6","crawleeVersion":"3.13.10","osType":"Darwin","nodeVersion":"v22.17.0"} -Extracted heading { level: 'h1', text: 'Your full‑stack platform for web scraping' } +Extracted heading { level: 'h1', text: 'Your full‑stack platform for web scraping' } Extracted heading { level: 'h3', text: 'TikTok Scraper' } Extracted heading { level: 'h3', text: 'Google Maps Scraper' } Extracted heading { level: 'h3', text: 'Instagram Scraper' } @@ -79,7 +79,7 @@ Extracted heading { level: 'h3', text: 'Instagram Scraper' } As you can see in the logs, the Actor extracts text from a web page. The main logic lives in `src/main.js`. Depending on your template, this file may be `src/main.ts` (TypeScript) or `src/main.py` (Python). -In the next step, we’ll explore the results in more detail. +In the next step, we'll explore the results in more detail. ### Step 3: Explore the Actor @@ -87,7 +87,7 @@ Let's explore the Actor structure. #### The `.actor` folder -The `.actor` folder contains the Actor configuration. The `actor.json` file defines the Actor's name, description, and other settings. Find more info in the [actor.json](https://docs.apify.com/platform/actors/development/actor-definition/actor-json) definition. +The `.actor` folder contains the Actor configuration. The `actor.json` file defines the Actor's name, description, and other settings. Find more info in the [actor.json](https://docs.apify.com/platform/using-actors/development/actor-definition/actor-json) definition. #### Actor's `input` @@ -101,11 +101,11 @@ This JSON Schema validates input automatically (no error handling needed), power ::: -Find more info in the [Input schema](/platform/actors/development/actor-definition/input-schema) documentation. +Find more info in the [Input schema](/platform/using-actors/development/actor-definition/input-schema) documentation. #### Actor's `storage` -The Actor system provides two storage types for files and results: [key-value](/platform/storage/key-value-store) store and [dataset](/platform/storage/dataset). +The Actor system provides two storage types for files and results: [key-value](/platform/core-concepts/storage/key-value-store) store and [dataset](/platform/core-concepts/storage/dataset). ##### Key-value store @@ -119,8 +119,8 @@ The dataset stores a series of data objects from web scraping, crawling, or data You define the Actor output using the Output schema files: -- [Dataset Schema Specification](/platform/actors/development/actor-definition/dataset-schema) -- [Key-value Store Schema Specification](/platform/actors/development/actor-definition/key-value-store-schema) +- [Dataset Schema Specification](/platform/using-actors/development/actor-definition/dataset-schema) +- [Key-value Store Schema Specification](/platform/using-actors/development/actor-definition/key-value-store-schema) The system uses this to generate an immutable JSON file that tells users where to find the Actor's results. @@ -154,5 +154,5 @@ Good job! 🎉 You're ready to develop your Actor. You can make changes to your - Visit the [Apify Academy](/academy) to access a comprehensive collection of tutorials, documentation, and learning resources. - To understand Actors in detail, read the [Actor Whitepaper](https://whitepaper.actor/). -- Check [Continuous integration](../deployment/continuous_integration.md) documentation to automate your Actor development process. -- After you finish building your first Actor, you can [share it with other users and even monetize it](../../publishing/index.mdx). +- Check [Continuous integration](/platform/using-actors/deployment/continuous-integration) documentation to automate your Actor development process. +- After you finish building your first Actor, you can [share it with other users and even monetize it](/platform/publishing-and-monetization/publish). diff --git a/sources/platform/actors/development/quick_start/start_web_ide.md b/sources/platform/1-getting-started/start_web_ide.md similarity index 94% rename from sources/platform/actors/development/quick_start/start_web_ide.md rename to sources/platform/1-getting-started/start_web_ide.md index 4028703076..58bafa89a7 100644 --- a/sources/platform/actors/development/quick_start/start_web_ide.md +++ b/sources/platform/1-getting-started/start_web_ide.md @@ -1,8 +1,8 @@ --- title: Web IDE -sidebar_position: 2 +sidebar_position: 3 description: Create your first Actor using the web IDE in Apify Console. -slug: /actors/development/quick-start/web-ide +slug: /getting-started/web-ide --- import Tabs from '@theme/Tabs'; @@ -153,5 +153,5 @@ Once you've made the desired changes, you can push the updated code back to the - Visit the [Apify Academy](/academy) to access a comprehensive collection of tutorials, documentation, and learning resources. - To understand Actors in detail, read the [Actor Whitepaper](https://whitepaper.actor/). -- Check [Continuous integration](../deployment/continuous_integration.md) documentation to automate your Actor development process. -- After you finish building your first Actor, you can [share it with other users and even monetize it](../../publishing/index.mdx). +- Check [Continuous integration](/platform/using-actors/deployment/continuous-integration) documentation to automate your Actor development process. +- After you finish building your first Actor, you can [share it with other users and even monetize it](/platform/publishing-and-monetization/publish). diff --git a/sources/platform/2-console/_category_.yml b/sources/platform/2-console/_category_.yml new file mode 100644 index 0000000000..1123a9e913 --- /dev/null +++ b/sources/platform/2-console/_category_.yml @@ -0,0 +1,2 @@ +label: 'Console' +position: 2 \ No newline at end of file diff --git a/sources/platform/console/billing.md b/sources/platform/2-console/billing.md similarity index 100% rename from sources/platform/console/billing.md rename to sources/platform/2-console/billing.md diff --git a/sources/platform/console/images/apify-console-homepage.png b/sources/platform/2-console/images/apify-console-homepage.png similarity index 100% rename from sources/platform/console/images/apify-console-homepage.png rename to sources/platform/2-console/images/apify-console-homepage.png diff --git a/sources/platform/console/images/console-account-two-factor-disabled.png b/sources/platform/2-console/images/console-account-two-factor-disabled.png similarity index 100% rename from sources/platform/console/images/console-account-two-factor-disabled.png rename to sources/platform/2-console/images/console-account-two-factor-disabled.png diff --git a/sources/platform/console/images/console-account-two-factor-enabled.png b/sources/platform/2-console/images/console-account-two-factor-enabled.png similarity index 100% rename from sources/platform/console/images/console-account-two-factor-enabled.png rename to sources/platform/2-console/images/console-account-two-factor-enabled.png diff --git a/sources/platform/console/images/console-actors-recently-used.png b/sources/platform/2-console/images/console-actors-recently-used.png similarity index 100% rename from sources/platform/console/images/console-actors-recently-used.png rename to sources/platform/2-console/images/console-actors-recently-used.png diff --git a/sources/platform/console/images/console-actors-runs.png b/sources/platform/2-console/images/console-actors-runs.png similarity index 100% rename from sources/platform/console/images/console-actors-runs.png rename to sources/platform/2-console/images/console-actors-runs.png diff --git a/sources/platform/console/images/console-billing-current-period.png b/sources/platform/2-console/images/console-billing-current-period.png similarity index 100% rename from sources/platform/console/images/console-billing-current-period.png rename to sources/platform/2-console/images/console-billing-current-period.png diff --git a/sources/platform/console/images/console-billing-historical-usage-by-actors.png b/sources/platform/2-console/images/console-billing-historical-usage-by-actors.png similarity index 100% rename from sources/platform/console/images/console-billing-historical-usage-by-actors.png rename to sources/platform/2-console/images/console-billing-historical-usage-by-actors.png diff --git a/sources/platform/console/images/console-billing-historical-usage.png b/sources/platform/2-console/images/console-billing-historical-usage.png similarity index 100% rename from sources/platform/console/images/console-billing-historical-usage.png rename to sources/platform/2-console/images/console-billing-historical-usage.png diff --git a/sources/platform/console/images/console-billing-invoices.png b/sources/platform/2-console/images/console-billing-invoices.png similarity index 100% rename from sources/platform/console/images/console-billing-invoices.png rename to sources/platform/2-console/images/console-billing-invoices.png diff --git a/sources/platform/console/images/console-billing-limits.png b/sources/platform/2-console/images/console-billing-limits.png similarity index 100% rename from sources/platform/console/images/console-billing-limits.png rename to sources/platform/2-console/images/console-billing-limits.png diff --git a/sources/platform/console/images/console-billing-pricing.png b/sources/platform/2-console/images/console-billing-pricing.png similarity index 100% rename from sources/platform/console/images/console-billing-pricing.png rename to sources/platform/2-console/images/console-billing-pricing.png diff --git a/sources/platform/console/images/console-billing-subscription.png b/sources/platform/2-console/images/console-billing-subscription.png similarity index 100% rename from sources/platform/console/images/console-billing-subscription.png rename to sources/platform/2-console/images/console-billing-subscription.png diff --git a/sources/platform/console/images/console-email-verification-page.png b/sources/platform/2-console/images/console-email-verification-page.png similarity index 100% rename from sources/platform/console/images/console-email-verification-page.png rename to sources/platform/2-console/images/console-email-verification-page.png diff --git a/sources/platform/console/images/console-forgotten-password-page.png b/sources/platform/2-console/images/console-forgotten-password-page.png similarity index 100% rename from sources/platform/console/images/console-forgotten-password-page.png rename to sources/platform/2-console/images/console-forgotten-password-page.png diff --git a/sources/platform/console/images/console-login.png b/sources/platform/2-console/images/console-login.png similarity index 100% rename from sources/platform/console/images/console-login.png rename to sources/platform/2-console/images/console-login.png diff --git a/sources/platform/console/images/console-reset-password-page.png b/sources/platform/2-console/images/console-reset-password-page.png similarity index 100% rename from sources/platform/console/images/console-reset-password-page.png rename to sources/platform/2-console/images/console-reset-password-page.png diff --git a/sources/platform/console/images/console-setup-two-factor-auth-key.png b/sources/platform/2-console/images/console-setup-two-factor-auth-key.png similarity index 100% rename from sources/platform/console/images/console-setup-two-factor-auth-key.png rename to sources/platform/2-console/images/console-setup-two-factor-auth-key.png diff --git a/sources/platform/console/images/console-sign-in-form.png b/sources/platform/2-console/images/console-sign-in-form.png similarity index 100% rename from sources/platform/console/images/console-sign-in-form.png rename to sources/platform/2-console/images/console-sign-in-form.png diff --git a/sources/platform/console/images/console-sign-in-methods-section.png b/sources/platform/2-console/images/console-sign-in-methods-section.png similarity index 100% rename from sources/platform/console/images/console-sign-in-methods-section.png rename to sources/platform/2-console/images/console-sign-in-methods-section.png diff --git a/sources/platform/console/images/console-sign-up-form.png b/sources/platform/2-console/images/console-sign-up-form.png similarity index 100% rename from sources/platform/console/images/console-sign-up-form.png rename to sources/platform/2-console/images/console-sign-up-form.png diff --git a/sources/platform/console/images/console-signup.png b/sources/platform/2-console/images/console-signup.png similarity index 100% rename from sources/platform/console/images/console-signup.png rename to sources/platform/2-console/images/console-signup.png diff --git a/sources/platform/console/images/console-store.png b/sources/platform/2-console/images/console-store.png similarity index 100% rename from sources/platform/console/images/console-store.png rename to sources/platform/2-console/images/console-store.png diff --git a/sources/platform/console/images/console-two-factor-app-setup.png b/sources/platform/2-console/images/console-two-factor-app-setup.png similarity index 100% rename from sources/platform/console/images/console-two-factor-app-setup.png rename to sources/platform/2-console/images/console-two-factor-app-setup.png diff --git a/sources/platform/console/images/console-two-factor-auth-disable.png b/sources/platform/2-console/images/console-two-factor-auth-disable.png similarity index 100% rename from sources/platform/console/images/console-two-factor-auth-disable.png rename to sources/platform/2-console/images/console-two-factor-auth-disable.png diff --git a/sources/platform/console/images/console-two-factor-authentication.png b/sources/platform/2-console/images/console-two-factor-authentication.png similarity index 100% rename from sources/platform/console/images/console-two-factor-authentication.png rename to sources/platform/2-console/images/console-two-factor-authentication.png diff --git a/sources/platform/console/images/console-two-factor-recovery-settings-request.png b/sources/platform/2-console/images/console-two-factor-recovery-settings-request.png similarity index 100% rename from sources/platform/console/images/console-two-factor-recovery-settings-request.png rename to sources/platform/2-console/images/console-two-factor-recovery-settings-request.png diff --git a/sources/platform/console/images/console-two-factor-recovery-settings-revealed.png b/sources/platform/2-console/images/console-two-factor-recovery-settings-revealed.png similarity index 100% rename from sources/platform/console/images/console-two-factor-recovery-settings-revealed.png rename to sources/platform/2-console/images/console-two-factor-recovery-settings-revealed.png diff --git a/sources/platform/console/images/console-two-factor-recovery-setup.png b/sources/platform/2-console/images/console-two-factor-recovery-setup.png similarity index 100% rename from sources/platform/console/images/console-two-factor-recovery-setup.png rename to sources/platform/2-console/images/console-two-factor-recovery-setup.png diff --git a/sources/platform/console/images/console-two-factor-use-recovery-code.png b/sources/platform/2-console/images/console-two-factor-use-recovery-code.png similarity index 100% rename from sources/platform/console/images/console-two-factor-use-recovery-code.png rename to sources/platform/2-console/images/console-two-factor-use-recovery-code.png diff --git a/sources/platform/console/index.md b/sources/platform/2-console/index.md similarity index 88% rename from sources/platform/console/index.md rename to sources/platform/2-console/index.md index dfb4ec056c..02db36b8f6 100644 --- a/sources/platform/console/index.md +++ b/sources/platform/2-console/index.md @@ -1,8 +1,7 @@ --- -title: Apify Console -description: Learn about Apify Console's easy account creation and user-friendly homepage for efficient web scraping management. +title: "Console" +description: "Web interface for managing Actors, runs, storage, and account settings" sidebar_position: 1 -category: platform slug: /console --- @@ -115,13 +114,13 @@ You can also navigate Apify Console via keyboard shortcuts. | Tab name | Description | |:---|:---| | [Apify Store](/platform/console/store)| Search for Actors that suit your web-scraping needs. | -| [Actors](/platform/actors)| View recent & bookmarked Actors. | -| [Runs](/platform/actors/running/runs-and-builds)| View your recent runs. | -| [Saved tasks](/platform/actors/running/tasks)| View your saved tasks. | -| [Schedules](/platform/schedules)| Schedule Actor runs & tasks to run at specified time. | +| [Actors](/platform/using-actors)| View recent & bookmarked Actors. | +| [Runs](/platform/using-actors/running/runs-and-builds)| View your recent runs. | +| [Saved tasks](/platform/using-actors/running/tasks)| View your saved tasks. | +| [Schedules](/platform/using-actors/schedules)| Schedule Actor runs & tasks to run at specified time. | | [Integrations](/platform/integrations)| View your integrations. | -| [Development](/platform/actors/development)| • My Actors - See Actors developed by you.
• Insights - see analytics for your Actors.
• Messaging - check on issues reported in your Actors or send emails to users of your Actors. | -| [Proxy](/platform/proxy)| View your proxy usage & credentials | -| [Storage](/platform/storage)| View stored results of your runs in various data formats. | +| [Development](/platform/building-actors)| • My Actors - See Actors developed by you.
• Insights - see analytics for your Actors.
• Messaging - check on issues reported in your Actors or send emails to users of your Actors. | +| [Proxy](/platform/core-concepts/proxy)| View your proxy usage & credentials | +| [Storage](/platform/core-concepts/storage)| View stored results of your runs in various data formats. | | [Billing](/platform/console/billing)| Billing information, statistics and invoices. | | [Settings](/platform/console/settings)| Settings of your account. | diff --git a/sources/platform/console/settings.md b/sources/platform/2-console/settings.md similarity index 100% rename from sources/platform/console/settings.md rename to sources/platform/2-console/settings.md diff --git a/sources/platform/console/store.md b/sources/platform/2-console/store.md similarity index 94% rename from sources/platform/console/store.md rename to sources/platform/2-console/store.md index fd38e87c49..0b80409745 100644 --- a/sources/platform/console/store.md +++ b/sources/platform/2-console/store.md @@ -25,4 +25,4 @@ You can also organize the results from the store by different criteria, includin Once you select an Actor from the store, you'll be directed to its specific page. Here, you can configure the settings for your future Actor run, save these configurations for later use, or run the Actor immediately. -For more information on Actors in Apify Store, visit our [Apify Store documentation](/sources/platform/actors/running/store.md). +For more information on Actors in Apify Store, visit our [Apify Store documentation](/platform/using-actors/actors-in-store). diff --git a/sources/platform/console/two-factor-authentication.md b/sources/platform/2-console/two-factor-authentication.md similarity index 100% rename from sources/platform/console/two-factor-authentication.md rename to sources/platform/2-console/two-factor-authentication.md diff --git a/sources/platform/3-using-actors/_category_.yml b/sources/platform/3-using-actors/_category_.yml new file mode 100644 index 0000000000..beb65f4d90 --- /dev/null +++ b/sources/platform/3-using-actors/_category_.yml @@ -0,0 +1,2 @@ +label: 'Using Actors' +position: 3 \ No newline at end of file diff --git a/sources/platform/actors/running/actor_standby.md b/sources/platform/3-using-actors/actor_standby.md similarity index 95% rename from sources/platform/actors/running/actor_standby.md rename to sources/platform/3-using-actors/actor_standby.md index b2c171b477..670fb5923d 100644 --- a/sources/platform/actors/running/actor_standby.md +++ b/sources/platform/3-using-actors/actor_standby.md @@ -2,7 +2,7 @@ title: Standby mode description: Use an Actor as a real-time API server. sidebar_position: 7.3 -slug: /actors/running/standby +slug: /using-actors/standby sidebar_label: Standby mode --- @@ -33,8 +33,8 @@ Generally speaking, Actors in Standby mode behave as standard HTTP servers. You ## How do I authenticate my requests -To authenticate requests to Actor Standby, follow the same process as [authenticating requests to the Apify API](../../integrations/programming/api.md). -You can provide your [API token](../../integrations/programming/api.md#api-token) in one of two ways: +To authenticate requests to Actor Standby, follow the same process as [authenticating requests to the Apify API](/platform/integrations/api). +You can provide your [API token](//platform/integrations/api#api-token) in one of two ways: 1. _Recommended_: Include the token in the `Authorization` header of your request as `Bearer `. This approach is recommended because it prevents your token from being logged in server logs. @@ -101,4 +101,4 @@ No, even if you use the Actor-level hostname with the default configuration, the ## How can I develop Actors using Standby mode -See the [Actor Standby development section](../development/programming_interface/actor_standby.md). +See the [Actor Standby development section](/platform/building-actors/programming-interface/standby). diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-export.png b/sources/platform/3-using-actors/images/actor-google-maps-scraper-export.png similarity index 100% rename from sources/platform/actors/running/images/actor-google-maps-scraper-export.png rename to sources/platform/3-using-actors/images/actor-google-maps-scraper-export.png diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-input.png b/sources/platform/3-using-actors/images/actor-google-maps-scraper-input.png similarity index 100% rename from sources/platform/actors/running/images/actor-google-maps-scraper-input.png rename to sources/platform/3-using-actors/images/actor-google-maps-scraper-input.png diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-results.png b/sources/platform/3-using-actors/images/actor-google-maps-scraper-results.png similarity index 100% rename from sources/platform/actors/running/images/actor-google-maps-scraper-results.png rename to sources/platform/3-using-actors/images/actor-google-maps-scraper-results.png diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-run.gif b/sources/platform/3-using-actors/images/actor-google-maps-scraper-run.gif similarity index 100% rename from sources/platform/actors/running/images/actor-google-maps-scraper-run.gif rename to sources/platform/3-using-actors/images/actor-google-maps-scraper-run.gif diff --git a/sources/platform/actors/running/images/actor-google-maps-scraper-running.png b/sources/platform/3-using-actors/images/actor-google-maps-scraper-running.png similarity index 100% rename from sources/platform/actors/running/images/actor-google-maps-scraper-running.png rename to sources/platform/3-using-actors/images/actor-google-maps-scraper-running.png diff --git a/sources/platform/actors/running/images/actor-results.png b/sources/platform/3-using-actors/images/actor-results.png similarity index 100% rename from sources/platform/actors/running/images/actor-results.png rename to sources/platform/3-using-actors/images/actor-results.png diff --git a/sources/platform/actors/running/images/actor-run.png b/sources/platform/3-using-actors/images/actor-run.png similarity index 100% rename from sources/platform/actors/running/images/actor-run.png rename to sources/platform/3-using-actors/images/actor-run.png diff --git a/sources/platform/actors/running/images/actor_standby/standby-tab.png b/sources/platform/3-using-actors/images/actor_standby/standby-tab.png similarity index 100% rename from sources/platform/actors/running/images/actor_standby/standby-tab.png rename to sources/platform/3-using-actors/images/actor_standby/standby-tab.png diff --git a/sources/platform/actors/running/images/apify-input.png b/sources/platform/3-using-actors/images/apify-input.png similarity index 100% rename from sources/platform/actors/running/images/apify-input.png rename to sources/platform/3-using-actors/images/apify-input.png diff --git a/sources/platform/actors/running/images/export-results.png b/sources/platform/3-using-actors/images/export-results.png similarity index 100% rename from sources/platform/actors/running/images/export-results.png rename to sources/platform/3-using-actors/images/export-results.png diff --git a/sources/platform/actors/running/images/input_and_output/actor-input.png b/sources/platform/3-using-actors/images/input_and_output/actor-input.png similarity index 100% rename from sources/platform/actors/running/images/input_and_output/actor-input.png rename to sources/platform/3-using-actors/images/input_and_output/actor-input.png diff --git a/sources/platform/actors/running/images/input_and_output/actor-options.png b/sources/platform/3-using-actors/images/input_and_output/actor-options.png similarity index 100% rename from sources/platform/actors/running/images/input_and_output/actor-options.png rename to sources/platform/3-using-actors/images/input_and_output/actor-options.png diff --git a/sources/platform/actors/running/images/input_and_output/actor-output.png b/sources/platform/3-using-actors/images/input_and_output/actor-output.png similarity index 100% rename from sources/platform/actors/running/images/input_and_output/actor-output.png rename to sources/platform/3-using-actors/images/input_and_output/actor-output.png diff --git a/sources/platform/actors/running/images/input_and_output/actor-storage.png b/sources/platform/3-using-actors/images/input_and_output/actor-storage.png similarity index 100% rename from sources/platform/actors/running/images/input_and_output/actor-storage.png rename to sources/platform/3-using-actors/images/input_and_output/actor-storage.png diff --git a/sources/platform/actors/running/images/runs_and_builds/actor-run-detail.png b/sources/platform/3-using-actors/images/runs_and_builds/actor-run-detail.png similarity index 100% rename from sources/platform/actors/running/images/runs_and_builds/actor-run-detail.png rename to sources/platform/3-using-actors/images/runs_and_builds/actor-run-detail.png diff --git a/sources/platform/actors/running/images/runs_and_builds/actor-run-options.png b/sources/platform/3-using-actors/images/runs_and_builds/actor-run-options.png similarity index 100% rename from sources/platform/actors/running/images/runs_and_builds/actor-run-options.png rename to sources/platform/3-using-actors/images/runs_and_builds/actor-run-options.png diff --git a/sources/platform/actors/running/images/store-google-maps-scraper.png b/sources/platform/3-using-actors/images/store-google-maps-scraper.png similarity index 100% rename from sources/platform/actors/running/images/store-google-maps-scraper.png rename to sources/platform/3-using-actors/images/store-google-maps-scraper.png diff --git a/sources/platform/actors/running/images/store.png b/sources/platform/3-using-actors/images/store.png similarity index 100% rename from sources/platform/actors/running/images/store.png rename to sources/platform/3-using-actors/images/store.png diff --git a/sources/platform/actors/running/images/store/apify_store_discounts_console.png b/sources/platform/3-using-actors/images/store/apify_store_discounts_console.png similarity index 100% rename from sources/platform/actors/running/images/store/apify_store_discounts_console.png rename to sources/platform/3-using-actors/images/store/apify_store_discounts_console.png diff --git a/sources/platform/actors/running/images/store/apify_store_discounts_full_table.png b/sources/platform/3-using-actors/images/store/apify_store_discounts_full_table.png similarity index 100% rename from sources/platform/actors/running/images/store/apify_store_discounts_full_table.png rename to sources/platform/3-using-actors/images/store/apify_store_discounts_full_table.png diff --git a/sources/platform/actors/running/images/store/apify_store_discounts_web.png b/sources/platform/3-using-actors/images/store/apify_store_discounts_web.png similarity index 100% rename from sources/platform/actors/running/images/store/apify_store_discounts_web.png rename to sources/platform/3-using-actors/images/store/apify_store_discounts_web.png diff --git a/sources/platform/actors/running/images/store/billing-paid-actors.png b/sources/platform/3-using-actors/images/store/billing-paid-actors.png similarity index 100% rename from sources/platform/actors/running/images/store/billing-paid-actors.png rename to sources/platform/3-using-actors/images/store/billing-paid-actors.png diff --git a/sources/platform/actors/running/images/store/console_pay_per_result_tag.png b/sources/platform/3-using-actors/images/store/console_pay_per_result_tag.png similarity index 100% rename from sources/platform/actors/running/images/store/console_pay_per_result_tag.png rename to sources/platform/3-using-actors/images/store/console_pay_per_result_tag.png diff --git a/sources/platform/actors/running/images/store/example_run_usage.png b/sources/platform/3-using-actors/images/store/example_run_usage.png similarity index 100% rename from sources/platform/actors/running/images/store/example_run_usage.png rename to sources/platform/3-using-actors/images/store/example_run_usage.png diff --git a/sources/platform/actors/running/images/store/free_vs_paid_actors.png b/sources/platform/3-using-actors/images/store/free_vs_paid_actors.png similarity index 100% rename from sources/platform/actors/running/images/store/free_vs_paid_actors.png rename to sources/platform/3-using-actors/images/store/free_vs_paid_actors.png diff --git a/sources/platform/actors/running/images/store/max-items-for-pay-per-result.png b/sources/platform/3-using-actors/images/store/max-items-for-pay-per-result.png similarity index 100% rename from sources/platform/actors/running/images/store/max-items-for-pay-per-result.png rename to sources/platform/3-using-actors/images/store/max-items-for-pay-per-result.png diff --git a/sources/platform/actors/running/images/store/paid-actors-billing.png b/sources/platform/3-using-actors/images/store/paid-actors-billing.png similarity index 100% rename from sources/platform/actors/running/images/store/paid-actors-billing.png rename to sources/platform/3-using-actors/images/store/paid-actors-billing.png diff --git a/sources/platform/actors/running/images/store/paid-actors-issues-tab.png b/sources/platform/3-using-actors/images/store/paid-actors-issues-tab.png similarity index 100% rename from sources/platform/actors/running/images/store/paid-actors-issues-tab.png rename to sources/platform/3-using-actors/images/store/paid-actors-issues-tab.png diff --git a/sources/platform/actors/running/images/store/paid-actors-store.png b/sources/platform/3-using-actors/images/store/paid-actors-store.png similarity index 100% rename from sources/platform/actors/running/images/store/paid-actors-store.png rename to sources/platform/3-using-actors/images/store/paid-actors-store.png diff --git a/sources/platform/actors/running/images/store/paid-actors-trial.png b/sources/platform/3-using-actors/images/store/paid-actors-trial.png similarity index 100% rename from sources/platform/actors/running/images/store/paid-actors-trial.png rename to sources/platform/3-using-actors/images/store/paid-actors-trial.png diff --git a/sources/platform/actors/running/images/store/pay-per-usage-actor-example.png b/sources/platform/3-using-actors/images/store/pay-per-usage-actor-example.png similarity index 100% rename from sources/platform/actors/running/images/store/pay-per-usage-actor-example.png rename to sources/platform/3-using-actors/images/store/pay-per-usage-actor-example.png diff --git a/sources/platform/actors/running/images/store/pay_per_event_example_actor.png b/sources/platform/3-using-actors/images/store/pay_per_event_example_actor.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_event_example_actor.png rename to sources/platform/3-using-actors/images/store/pay_per_event_example_actor.png diff --git a/sources/platform/actors/running/images/store/pay_per_event_historical_usage_tab.png b/sources/platform/3-using-actors/images/store/pay_per_event_historical_usage_tab.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_event_historical_usage_tab.png rename to sources/platform/3-using-actors/images/store/pay_per_event_historical_usage_tab.png diff --git a/sources/platform/actors/running/images/store/pay_per_event_max_charge_per_run.png b/sources/platform/3-using-actors/images/store/pay_per_event_max_charge_per_run.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_event_max_charge_per_run.png rename to sources/platform/3-using-actors/images/store/pay_per_event_max_charge_per_run.png diff --git a/sources/platform/actors/running/images/store/pay_per_event_price_on_run_detail.png b/sources/platform/3-using-actors/images/store/pay_per_event_price_on_run_detail.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_event_price_on_run_detail.png rename to sources/platform/3-using-actors/images/store/pay_per_event_price_on_run_detail.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_actor_items.png b/sources/platform/3-using-actors/images/store/pay_per_result_actor_items.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_actor_items.png rename to sources/platform/3-using-actors/images/store/pay_per_result_actor_items.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_actor_store_card.png b/sources/platform/3-using-actors/images/store/pay_per_result_actor_store_card.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_actor_store_card.png rename to sources/platform/3-using-actors/images/store/pay_per_result_actor_store_card.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_billing_usage_section.png b/sources/platform/3-using-actors/images/store/pay_per_result_billing_usage_section.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_billing_usage_section.png rename to sources/platform/3-using-actors/images/store/pay_per_result_billing_usage_section.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_max_items.png b/sources/platform/3-using-actors/images/store/pay_per_result_max_items.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_max_items.png rename to sources/platform/3-using-actors/images/store/pay_per_result_max_items.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_run_detail.png b/sources/platform/3-using-actors/images/store/pay_per_result_run_detail.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_run_detail.png rename to sources/platform/3-using-actors/images/store/pay_per_result_run_detail.png diff --git a/sources/platform/actors/running/images/store/pay_per_result_run_list.png b/sources/platform/3-using-actors/images/store/pay_per_result_run_list.png similarity index 100% rename from sources/platform/actors/running/images/store/pay_per_result_run_list.png rename to sources/platform/3-using-actors/images/store/pay_per_result_run_list.png diff --git a/sources/platform/actors/running/images/store/rental-actor-example.png b/sources/platform/3-using-actors/images/store/rental-actor-example.png similarity index 100% rename from sources/platform/actors/running/images/store/rental-actor-example.png rename to sources/platform/3-using-actors/images/store/rental-actor-example.png diff --git a/sources/platform/actors/running/images/store/rental-actor-trial-example.png b/sources/platform/3-using-actors/images/store/rental-actor-trial-example.png similarity index 100% rename from sources/platform/actors/running/images/store/rental-actor-trial-example.png rename to sources/platform/3-using-actors/images/store/rental-actor-trial-example.png diff --git a/sources/platform/actors/running/images/tasks/tasks-create-configure.png b/sources/platform/3-using-actors/images/tasks/tasks-create-configure.png similarity index 100% rename from sources/platform/actors/running/images/tasks/tasks-create-configure.png rename to sources/platform/3-using-actors/images/tasks/tasks-create-configure.png diff --git a/sources/platform/actors/running/images/tasks/tasks-create-task.png b/sources/platform/3-using-actors/images/tasks/tasks-create-task.png similarity index 100% rename from sources/platform/actors/running/images/tasks/tasks-create-task.png rename to sources/platform/3-using-actors/images/tasks/tasks-create-task.png diff --git a/sources/platform/actors/running/images/tasks/tasks-start-after-configuration.png b/sources/platform/3-using-actors/images/tasks/tasks-start-after-configuration.png similarity index 100% rename from sources/platform/actors/running/images/tasks/tasks-start-after-configuration.png rename to sources/platform/3-using-actors/images/tasks/tasks-start-after-configuration.png diff --git a/sources/platform/actors/running/images/tasks/tasks-start-button.png b/sources/platform/3-using-actors/images/tasks/tasks-start-button.png similarity index 100% rename from sources/platform/actors/running/images/tasks/tasks-start-button.png rename to sources/platform/3-using-actors/images/tasks/tasks-start-button.png diff --git a/sources/platform/actors/running/images/usage_and_resources/actor-usage.png b/sources/platform/3-using-actors/images/usage_and_resources/actor-usage.png similarity index 100% rename from sources/platform/actors/running/images/usage_and_resources/actor-usage.png rename to sources/platform/3-using-actors/images/usage_and_resources/actor-usage.png diff --git a/sources/platform/actors/running/images/usage_and_resources/memory-cpu-usage-spike.png b/sources/platform/3-using-actors/images/usage_and_resources/memory-cpu-usage-spike.png similarity index 100% rename from sources/platform/actors/running/images/usage_and_resources/memory-cpu-usage-spike.png rename to sources/platform/3-using-actors/images/usage_and_resources/memory-cpu-usage-spike.png diff --git a/sources/platform/actors/running/images/usage_and_resources/memory-settings.png b/sources/platform/3-using-actors/images/usage_and_resources/memory-settings.png similarity index 100% rename from sources/platform/actors/running/images/usage_and_resources/memory-settings.png rename to sources/platform/3-using-actors/images/usage_and_resources/memory-settings.png diff --git a/sources/platform/actors/running/images/usage_and_resources/usage-and-resources-runs-usage-details.png b/sources/platform/3-using-actors/images/usage_and_resources/usage-and-resources-runs-usage-details.png similarity index 100% rename from sources/platform/actors/running/images/usage_and_resources/usage-and-resources-runs-usage-details.png rename to sources/platform/3-using-actors/images/usage_and_resources/usage-and-resources-runs-usage-details.png diff --git a/sources/platform/actors/running/images/usage_and_resources/usage-and-resources-runs-usage.png b/sources/platform/3-using-actors/images/usage_and_resources/usage-and-resources-runs-usage.png similarity index 100% rename from sources/platform/actors/running/images/usage_and_resources/usage-and-resources-runs-usage.png rename to sources/platform/3-using-actors/images/usage_and_resources/usage-and-resources-runs-usage.png diff --git a/sources/platform/actors/running/index.md b/sources/platform/3-using-actors/index.md similarity index 95% rename from sources/platform/actors/running/index.md rename to sources/platform/3-using-actors/index.md index 56541d822d..5731ccfc70 100644 --- a/sources/platform/actors/running/index.md +++ b/sources/platform/3-using-actors/index.md @@ -1,8 +1,8 @@ --- -title: Running Actors -description: Start an Actor from Apify Console or via API. Learn about Actor lifecycles, how to specify settings and version, provide input, and resurrect finished runs. -sidebar_position: 7.1 -slug: /actors/running +title: "Using Actors" +description: "Run and manage Actors from the Apify Store" +sidebar_position: 1 +slug: /using-actors --- **In this section, you learn how to run Apify Actors using Apify Console or programmatically. You will learn about their configuration, versioning, data retention, usage, and pricing.** diff --git a/sources/platform/actors/running/input_and_output.md b/sources/platform/3-using-actors/input_and_output.md similarity index 89% rename from sources/platform/actors/running/input_and_output.md rename to sources/platform/3-using-actors/input_and_output.md index 1a9277630c..c5ea940a60 100644 --- a/sources/platform/actors/running/input_and_output.md +++ b/sources/platform/3-using-actors/input_and_output.md @@ -2,7 +2,7 @@ title: Input and output description: Configure your Actor's input parameters using Apify Console, locally or via API. Access parameters in key-value stores from your Actor's code. sidebar_position: 2 -slug: /actors/running/input-and-output +slug: /using-actors/input-and-output --- **Configure your Actor's input parameters using Apify Console, locally or via API. Access parameters in key-value stores from your Actor's code.** @@ -29,7 +29,7 @@ When running an Actor using the [API](https://docs.apify.com/api/v2) you can pas ### Options - Build, Timeout, and Memory -As part of the input, you can also specify run options such as [Build](../development/builds_and_runs/builds.md), Timeout, and [Memory](./usage_and_resources.md) for your Actor run. +As part of the input, you can also specify run options such as [Build](///platform/building-actors/builds-and-runs/builds), Timeout, and [Memory](///platform/using-actors/usage-and-resources) for your Actor run. ![Run options](./images/input_and_output/actor-options.png) @@ -44,7 +44,7 @@ As part of the input, you can also specify run options such as [Build](../develo While the input object provides a way to instruct Actors, an Actor can also generate an output, usually stored in its default [Dataset](../../storage/dataset), but some additional files might be stored in its [Key-value store](../../storage/key-value-store). Always read the Actor's README to learn more about its output. -For more details about storages, visit the [Storage](../../storage/index.md) section. +For more details about storages, visit the [Storage](/platform/core-concepts/storage) section. You can quickly access the Actor's output from the run detail page: diff --git a/sources/platform/actors/running/runs_and_builds.md b/sources/platform/3-using-actors/runs_and_builds.md similarity index 97% rename from sources/platform/actors/running/runs_and_builds.md rename to sources/platform/3-using-actors/runs_and_builds.md index 9accc07dae..8df9baec40 100644 --- a/sources/platform/actors/running/runs_and_builds.md +++ b/sources/platform/3-using-actors/runs_and_builds.md @@ -2,7 +2,7 @@ title: Runs and builds description: Learn about Actor builds and runs, their lifecycle, sharing, and data retention policy. sidebar_position: 2 -slug: /actors/running/runs-and-builds +slug: /using-actors/runs-and-builds --- **Learn about Actor builds and runs, their lifecycle, sharing, and data retention policy.** @@ -25,7 +25,7 @@ Each build may have different features, input, or output. By fixing the build to ## Runs -When you start an Actor, an Actor run is created. An Actor run is a Docker container created from the build's Docker image with dedicated resources (CPU, memory, disk space). For more on this topic, see [Usage and resources](./usage_and_resources.md). +When you start an Actor, an Actor run is created. An Actor run is a Docker container created from the build's Docker image with dedicated resources (CPU, memory, disk space). For more on this topic, see [Usage and resources](///platform/using-actors/usage-and-resources). Each run has its own (default) [storages](../../storage) assigned, which it may but not necessarily need to use: @@ -129,4 +129,4 @@ Apify securely stores your ten most recent runs indefinitely, ensuring your reco ## Sharing -Share your Actor runs with other Apify users via the [access rights](../../collaboration/index.md) system. +Share your Actor runs with other Apify users via the [access rights](/platform/collaboration) system. diff --git a/sources/platform/schedules.md b/sources/platform/3-using-actors/schedules.md similarity index 88% rename from sources/platform/schedules.md rename to sources/platform/3-using-actors/schedules.md index ef13cc0048..942371e46d 100644 --- a/sources/platform/schedules.md +++ b/sources/platform/3-using-actors/schedules.md @@ -1,9 +1,8 @@ --- -title: Schedules -description: Learn how to automatically start your Actor and task runs and the basics of cron expressions. Set up and manage your schedules from Apify Console or via API. -sidebar_position: 8 -category: platform -slug: /schedules +title: "Schedules" +description: "Learn how to automatically start your Actor and task runs and the basics of cron expressions. Set up and manage your schedules from Apify Console or via API." +sidebar_position: 3 +slug: /using-actors/schedules --- **Learn how to automatically start your Actor and task runs and the basics of cron expressions. Set up and manage your schedules from Apify Console or via API.** @@ -34,7 +33,7 @@ Each schedule can be associated with a maximum of _10_ Actors and _10_ Actor tas ## Setting up a new schedule -Before setting up a new schedule, you should have the [Actor](./actors/index.mdx) or [task](./actors/running/tasks.md) you want to schedule prepared and tested. +Before setting up a new schedule, you should have the Actor or [task](//platform/using-actors/tasks) you want to schedule prepared and tested. To schedule an Actor, you need to have run it at least once before. To run the Actor, navigate to the Actor's page through [Apify Console](https://console.apify.com/store), where you can configure and initiate the Actor's run with your preferred settings by clicking the **Start** button. After this initial run, you can then use Schedules to automate future runs. @@ -50,20 +49,20 @@ Click on the name (by default it is **My Schedule**), there you can change its n You can adjust how often your Actor or task runs using the [schedule setup tool](#schedule-setup). You can find it by clicking on the **Schedule setup** card. -![New schedule](./images/schedules-overview.png) +![New schedule](../images/schedules-overview.png) Next, you'll need to give the schedule something to run. This is where the Actor or task you prepared earlier comes in. Click on the **Add** dropdown and select whether you want to schedule an Actor or task. -If you're scheduling an Actor run, you'll be able to specify the Actor's [input](./actors/running/input_and_output.md) and running options like [build](./actors/development/builds_and_runs/builds.md), timeout, [memory](./actors/running/usage_and_resources.md). +If you're scheduling an Actor run, you'll be able to specify the Actor's [input](//platform/using-actors/input-and-output) and running options like [build](//platform/building-actors/builds-and-runs/builds), timeout, [memory](//platform/using-actors/usage-and-resources). The **timeout** value is specified in seconds; a value of _0_ means there is no timeout, and the Actor runs until it finishes. If you don't provide an input, then the Actor's default input is used. If you provide an input with some fields missing, the missing fields are filled in with values from the default input. If input options are not provided, the default options values are used. -![Add Actor to schedule](./images/schedules-actor-input.png) +![Add Actor to schedule](../images/schedules-actor-input.png) If you're scheduling a task, just select the task you prepared earlier using the drop-down. If you need to override the task's input, you can pass it as a JSON object in the **Input JSON overrides** field. -![Add task to schedule](./images/schedules-task-input.png) +![Add task to schedule](../images/schedules-task-input.png) To add more Actors or tasks, just repeat the process. @@ -75,7 +74,7 @@ For integrations, you can also add a [webhook](/platform/integrations/webhooks) To create a new [schedule](/api/v2/schedules) using the Apify API, send a `POST` request to the [create schedule](/api/v2/schedules-post) endpoint. -You can find your [secret API token](./integrations/index.mdx) under the [Integrations](https://console.apify.com/account?tab=integrations) tab of your Apify account settings. +You can find your [secret API token](/platform/integrations) under the [Integrations](https://console.apify.com/account?tab=integrations) tab of your Apify account settings. :::caution API authentication recommendations When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL ([more info](/api/v2#authentication)). @@ -116,7 +115,7 @@ For more information, refer to the [schedules](/api/v2/schedule-get) section in The schedule setup tool uses [cron expressions](https://en.wikipedia.org/wiki/Cron#CRON_expression) to specify run times. If you're familiar with how to use them and need a specific run schedule, you can dive right in. If not, don't worry - the setup tool has a visual custom schedule builder that provides a similar level of control as cron expressions, though it's not quite as powerful. -![Schedule setup tool](./images/schedules-setup-tool.png) +![Schedule setup tool](../images/schedules-setup-tool.png) The **Next runs** section shows when the next run will be, if you click on **Show more** button it will expand and show you the next five runs. You can use this live feedback to experiment until you find the correct configuration. @@ -128,7 +127,7 @@ The schedule setup tool allows you to control the schedule's notifications. All If you want to manage the notifications for your schedules in bulk, you can do this from the [Notifications](https://console.apify.com/settings/notifications) settings tab. As long as you have 15 schedules or less, you can manage their notifications all at once by clicking the **Manage notifications for specific schedules** button. -![Schedule bulk notifications](./images/schedules-bulk-notifications.png) +![Schedule bulk notifications](../images/schedules-bulk-notifications.png) ## Cron expressions diff --git a/sources/platform/actors/running/store.md b/sources/platform/3-using-actors/store.md similarity index 90% rename from sources/platform/actors/running/store.md rename to sources/platform/3-using-actors/store.md index 4b4b47f0ff..2c1a31b6a0 100644 --- a/sources/platform/actors/running/store.md +++ b/sources/platform/3-using-actors/store.md @@ -2,7 +2,7 @@ title: Actors in Store description: Apify Store is home to thousands of public Actors available to the Apify community. It's the easiest way for you to start with Apify. sidebar_position: 1 -slug: /actors/running/actors-in-store +slug: /using-actors/actors-in-store #display H2 to H4 heading toc_min_heading_level: 2 toc_max_heading_level: 4 @@ -14,7 +14,7 @@ toc_max_heading_level: 4 :::info Publishing and monetizing Actors -Anyone is welcome to [publish Actors](/platform/actors/publishing) in the store, and you can even [monetize your Actors](/platform/actors/publishing/monetize). For more information about how to monetize your Actor, best practices, SEO, and promotion tips and tricks, head over to the [Getting the most of your public Actors in Apify Store](/academy/actor-marketing-playbook) section of the Apify Developers Academy. +Anyone is welcome to [publish Actors](/platform/publishing-and-monetization) in the store, and you can even [monetize your Actors](/platform/publishing-and-monetization/monetize). For more information about how to monetize your Actor, best practices, SEO, and promotion tips and tricks, head over to the [Getting the most of your public Actors in Apify Store](/academy/actor-marketing-playbook) section of the Apify Developers Academy. ::: @@ -64,7 +64,7 @@ _Example_: You activate a 7-day trial of an Actor at _noon of April 1, 2021_. If ##### How am I charged for Actor rental? -The rental fee for an Actor is automatically subtracted from your prepaid platform usage, similarly to, e.g. [compute units](./usage_and_resources.md). If you don't have enough usage prepaid, you will need to cover any overage in the next invoice. +The rental fee for an Actor is automatically subtracted from your prepaid platform usage, similarly to, e.g. [compute units](///platform/using-actors/usage-and-resources). If you don't have enough usage prepaid, you will need to cover any overage in the next invoice. ##### Will I be automatically charged at the end of the free trial? @@ -169,7 +169,7 @@ You would still pay for the long term storage of results, same as for pay per re #### Some Actors declare that I still need to pay for usage, how come? -When an Actor operates in [Standby mode](https://docs.apify.com/platform/actors/running/standby), you control how the background runs scale and how efficiently they are utilized. In this case, you are responsible for paying the platform usage costs of these runs in addition to the event charges. Some selected Standby Actors may have this usage component waived, so always check the pricing page of the specific Actor to determine whether you will be charged for usage or only for events. +When an Actor operates in [Standby mode](https://docs.apify.com/platform/using-actors/running/standby), you control how the background runs scale and how efficiently they are utilized. In this case, you are responsible for paying the platform usage costs of these runs in addition to the event charges. Some selected Standby Actors may have this usage component waived, so always check the pricing page of the specific Actor to determine whether you will be charged for usage or only for events. #### Where do I see how much I was charged for the pay per result Actors? @@ -193,7 +193,7 @@ Please, in such a case, do not hesitate to contact the Actor author or our suppo ### Pay per usage -When you use a pay per usage Actor, you are only charged for the platform usage that the runs of this Actor generate. [Platform usage](./usage_and_resources.md) includes components such as compute units, operations on [storages](/platform/storage), and usage of [residential proxies](/platform/proxy/residential-proxy) or [SERPs](/platform/proxy/google-serp-proxy). +When you use a pay per usage Actor, you are only charged for the platform usage that the runs of this Actor generate. [Platform usage](///platform/using-actors/usage-and-resources) includes components such as compute units, operations on [storages](/platform/core-concepts/storage), and usage of [residential proxies](/platform/core-concepts/proxy/residential-proxy) or [SERPs](/platform/core-concepts/proxy/google-serp-proxy). ![Pay for usage Actor example](./images/store/pay-per-usage-actor-example.png) @@ -201,7 +201,7 @@ When you use a pay per usage Actor, you are only charged for the platform usage With this model, it's very easy to see how many platform resources each Actor run consumed, but it is quite difficult to estimate their usage beforehand. The best way to find the costs of free Actors upfront is to try out the Actor on a limited scope (for example, on a small number of pages) and evaluate the consumption. You can easily do that using our [free plan](https://apify.com/pricing). -_For more information on platform usage cost see the [usage and resources](./usage_and_resources.md) page._ +_For more information on platform usage cost see the [usage and resources](///platform/using-actors/usage-and-resources) page._ ::: diff --git a/sources/platform/actors/running/tasks.md b/sources/platform/3-using-actors/tasks.md similarity index 76% rename from sources/platform/actors/running/tasks.md rename to sources/platform/3-using-actors/tasks.md index 132ebb7b67..8dc1b7f090 100644 --- a/sources/platform/actors/running/tasks.md +++ b/sources/platform/3-using-actors/tasks.md @@ -2,7 +2,7 @@ title: Actor tasks description: Create and save reusable configurations of Apify Actors tailored to specific use cases. sidebar_position: 7.2 -slug: /actors/running/tasks +slug: /using-actors/tasks sidebar_label: Tasks --- @@ -10,7 +10,7 @@ sidebar_label: Tasks --- -Actor tasks let you create multiple reusable configurations of a single Actor, adapted for specific use cases. For example, you can create one [_Web Scraper_](https://apify.com/apify/web-scraper) configuration (task) that scrapes the latest reviews from imdb.com, another that scrapes nike.com for the latest sneakers, and a third that scrapes your competitor's e-shop. You can then use and reuse these configurations directly from [Apify Console](https://console.apify.com/actors/tasks), [Schedules](../../schedules.md), or [API](/api/v2/actor-task-runs-post). +Actor tasks let you create multiple reusable configurations of a single Actor, adapted for specific use cases. For example, you can create one [_Web Scraper_](https://apify.com/apify/web-scraper) configuration (task) that scrapes the latest reviews from IMDb, another that scrapes nike.com for the latest sneakers, and a third that scrapes your competitor's e-shop. You can then use and reuse these configurations directly from [Apify Console](https://console.apify.com/actors/tasks), [Schedules](/platform/using-actors/schedules), or [API](/api/v2/actor-task-runs-post). You can find all your tasks in the [Apify Console](https://console.apify.com/actors/tasks). @@ -28,7 +28,7 @@ You can set up your task's input under the **Input** tab. A task's input configu An Actors' input fields may vary depending on their purpose, but they all follow the same principle: _you provide an Actor with the information it needs so it can do what you want it to do._ -You can set run options such as timeout and [memory](./usage_and_resources.md) in the **Run options** tab of the task's input configuration. +You can set run options such as timeout and [memory](///platform/using-actors/usage-and-resources) in the **Run options** tab of the task's input configuration. ### Naming @@ -46,11 +46,11 @@ Or using the **Start** button positioned following the input configuration. You can also run tasks using: -- [Schedules](../../schedules.md). +- [Schedules](/platform/using-actors/schedules). - Directly via the [Apify API](/api/v2/actor-task-runs-post). - The [JavaScript API client](/api/client/js/reference/class/TaskClient). - The [Python API client](/api/client/python/reference/class/TaskClient). ## Share -Like any other resource, you can share your Actor tasks with other Apify users via the [access rights](../../collaboration/index.md) system. +Like any other resource, you can share your Actor tasks with other Apify users via the [access rights](/platform/collaboration) system. diff --git a/sources/platform/actors/running/usage_and_resources.md b/sources/platform/3-using-actors/usage_and_resources.md similarity index 93% rename from sources/platform/actors/running/usage_and_resources.md rename to sources/platform/3-using-actors/usage_and_resources.md index 5a42b7675b..fd6180a9c7 100644 --- a/sources/platform/actors/running/usage_and_resources.md +++ b/sources/platform/3-using-actors/usage_and_resources.md @@ -2,7 +2,7 @@ title: Usage and resources description: Learn about your Actors' memory and processing power requirements, their relationship with Docker resources, minimum requirements for different use cases and its impact on the cost. sidebar_position: 2 -slug: /actors/running/usage-and-resources +slug: /using-actors/usage-and-resources --- **Learn about your Actors' memory and processing power requirements, their relationship with Docker resources, minimum requirements for different use cases and its impact on the cost.** @@ -11,13 +11,13 @@ slug: /actors/running/usage-and-resources ## Resources -[Actors](../index.mdx) run in [Docker containers](https://www.docker.com/resources/what-container/), which have a [limited amount of resources](https://phoenixnap.com/kb/docker-memory-and-cpu-limit) (memory, CPU, disk size, etc). When starting, the Actor needs to be allocated a certain share of those resources, such as CPU capacity that is necessary for the Actor to run. +[Actors](/platform) run in [Docker containers](https://www.docker.com/resources/what-container/), which have a [limited amount of resources](https://phoenixnap.com/kb/docker-memory-and-cpu-limit) (memory, CPU, disk size, etc). When starting, the Actor needs to be allocated a certain share of those resources, such as CPU capacity that is necessary for the Actor to run. ![Setting an Actor's memory](./images/usage_and_resources/memory-settings.png) Assigning an Actor a specific **Memory** capacity, also determines the allocated CPU power and its disk size. -Check out the [Limits](../../limits.md) page for detailed information on Actor memory, CPU limits, disk size and other limits. +Check out the [Limits](/platform/limits) page for detailed information on Actor memory, CPU limits, disk size and other limits. ### Memory @@ -28,7 +28,7 @@ When invoking an Actor, the caller must specify the memory allocation for the Ac - The maximum allowed value is `32768MB` - Acceptable values include: `128MB`, `256MB`, `512MB`, `1024MB`, `2048MB`, `4096MB`, `8192MB`, `16384MB`, and `32768MB` -Additionally, each user has a certain total limit of memory for running Actors. The sum of memory allocated for all running Actors and builds needs to be within this limit, otherwise the user cannot start a new Actor. For more details, see [limits](../../limits.md). +Additionally, each user has a certain total limit of memory for running Actors. The sum of memory allocated for all running Actors and builds needs to be within this limit, otherwise the user cannot start a new Actor. For more details, see [limits](/platform/limits). ### CPU diff --git a/sources/platform/4-core-concepts/_category_.yml b/sources/platform/4-core-concepts/_category_.yml new file mode 100644 index 0000000000..62802a584c --- /dev/null +++ b/sources/platform/4-core-concepts/_category_.yml @@ -0,0 +1,2 @@ +label: 'Core concepts' +position: 4 \ No newline at end of file diff --git a/sources/platform/4-core-concepts/index.md b/sources/platform/4-core-concepts/index.md new file mode 100644 index 0000000000..bb8e74ffde --- /dev/null +++ b/sources/platform/4-core-concepts/index.md @@ -0,0 +1,61 @@ +--- +title: Core concepts +description: "Essential platform features including storage for data management and proxies for anonymous web access across different geographies." +sidebar_position: 1 +slug: /core-concepts +--- + +import Card from "@site/src/components/Card"; +import CardGrid from "@site/src/components/CardGrid"; + +**Essential platform features including storage for data management and proxies for anonymous web access across different geographies.** + +--- + +The Apify platform provides core infrastructure services that enable effective web scraping, automation, and data processing workflows. + +## Storage + +The platform provides three types of storage accessible both within the [Apify Console](https://console.apify.com/storage) and externally through the [REST API](/api/v2), [API clients](/api), or [SDKs](/sdk). + + + + + + + +## Proxies + +[Apify Proxy](https://apify.com/proxy) allows you to change your IP address when web scraping to reduce the chance of being [blocked](/academy/anti-scraping/techniques) because of your geographical location. The proxy service monitors the health of your IP pool and intelligently rotates addresses to prevent IP address-based blocking. + +You can use proxies in your [Actors](/platform/using-actors) or any other application that supports HTTP proxies. View your proxy settings and password on the [Proxy](https://console.apify.com/proxy) page in Apify Console. For pricing information, visit [apify.com/pricing](https://apify.com/pricing). + + + + + + diff --git a/sources/platform/4-core-concepts/proxy/_category_.yml b/sources/platform/4-core-concepts/proxy/_category_.yml new file mode 100644 index 0000000000..710e5d6f56 --- /dev/null +++ b/sources/platform/4-core-concepts/proxy/_category_.yml @@ -0,0 +1,2 @@ +label: 'Proxies' +position: 3 diff --git a/sources/platform/proxy/datacenter_proxy.md b/sources/platform/4-core-concepts/proxy/datacenter_proxy.md similarity index 98% rename from sources/platform/proxy/datacenter_proxy.md rename to sources/platform/4-core-concepts/proxy/datacenter_proxy.md index 1cf81cb789..3445c58ae5 100644 --- a/sources/platform/proxy/datacenter_proxy.md +++ b/sources/platform/4-core-concepts/proxy/datacenter_proxy.md @@ -2,7 +2,7 @@ title: Datacenter proxy description: Learn how to reduce blocking when web scraping using IP address rotation. See proxy parameters and learn to implement Apify Proxy in an application. sidebar_position: 10.2 -slug: /proxy/datacenter-proxy +slug: /core-concepts/proxy/datacenter-proxy --- import Tabs from '@theme/Tabs'; @@ -190,7 +190,7 @@ If you use the session at least once a day, it will never expire, with two possi If the session is discarded due to the reasons above, it is assigned a new IP address. -To learn more about [sessions](./usage.md#sessions) and [IP address rotation](./usage.md#ip-address-rotation), see the [proxy overview page](./index.md). +To learn more about [sessions](/platform/core-concepts/proxy/usage#sessions) and [IP address rotation](/platform/core-concepts/proxy/usage#ip-address-rotation), see the [proxy overview page](/platform/core-concepts/proxy). ### Examples using sessions diff --git a/sources/platform/proxy/google_serp_proxy.md b/sources/platform/4-core-concepts/proxy/google_serp_proxy.md similarity index 87% rename from sources/platform/proxy/google_serp_proxy.md rename to sources/platform/4-core-concepts/proxy/google_serp_proxy.md index bba4ecb9d0..1e30e887a1 100644 --- a/sources/platform/proxy/google_serp_proxy.md +++ b/sources/platform/4-core-concepts/proxy/google_serp_proxy.md @@ -2,7 +2,7 @@ title: Google SERP proxy description: Learn how to collect search results from Google Search-powered tools. Get search results from localized domains in multiple countries, e.g. the US and Germany. sidebar_position: 10.4 -slug: /proxy/google-serp-proxy +slug: /core-concepts/proxy/google-serp-proxy --- import Tabs from '@theme/Tabs'; @@ -42,9 +42,9 @@ When using Google SERP proxy, the username should always be: groups-GOOGLE_SERP ``` -Unlike [datacenter](./datacenter_proxy.md) or [residential](./residential_proxy.md) proxies, there is no [session](./usage.md#sessions) parameter. +Unlike [datacenter](/platform/core-concepts/proxy/datacenter-proxy) or [residential](/platform/core-concepts/proxy/residential-proxy) proxies, there is no [session](/platform/core-concepts/proxy/usage#sessions) parameter. -If you use the `country` [parameter](./usage.md), the Google proxy location is used if you access a website whose hostname (stripped of `www.`) starts with **google**. +If you use the `country` [parameter](/platform/core-concepts/proxy/usage), the Google proxy location is used if you access a website whose hostname (stripped of `www.`) starts with **google**. ## Country selection @@ -63,7 +63,7 @@ See a [full list](https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXW ### Using the Apify SDK -If you are developing your own Apify [Actor](../actors/index.mdx) using the [Apify SDK](/sdk) and [Crawlee](https://crawlee.dev/), the most efficient way to use Google SERP proxy is [CheerioCrawler](https://crawlee.dev/api/cheerio-crawler/class/CheerioCrawler). This is because Google SERP proxy [only returns a page's HTML](./index.md). Alternatively, you can use the [got-scraping](https://github.com/apify/got-scraping) [NPM package](https://www.npmjs.com/package/got-scraping) by specifying the proxy URL in the options. For Python, you can leverage the [`requests`](https://pypi.org/project/requests/) library along with the Apify SDK. +If you are developing your own Apify [Actor](/platform/using-actors) using the [Apify SDK](/sdk) and [Crawlee](https://crawlee.dev/), the most efficient way to use Google SERP proxy is [CheerioCrawler](https://crawlee.dev/api/cheerio-crawler/class/CheerioCrawler). This is because Google SERP proxy [only returns a page's HTML](/platform/core-concepts/proxy). Alternatively, you can use the [got-scraping](https://github.com/apify/got-scraping) [npm package](https://www.npmjs.com/package/got-scraping) by specifying the proxy URL in the options. For Python, you can leverage the [`requests`](https://pypi.org/project/requests/) library along with the Apify SDK. The following examples get a list of search results for the keyword **wikipedia** from the USA (`google.com`). diff --git a/sources/platform/proxy/images/proxy-status.png b/sources/platform/4-core-concepts/proxy/images/proxy-status.png similarity index 100% rename from sources/platform/proxy/images/proxy-status.png rename to sources/platform/4-core-concepts/proxy/images/proxy-status.png diff --git a/sources/platform/proxy/residential_proxy.md b/sources/platform/4-core-concepts/proxy/residential_proxy.md similarity index 79% rename from sources/platform/proxy/residential_proxy.md rename to sources/platform/4-core-concepts/proxy/residential_proxy.md index 18fa525984..73a898b4a7 100644 --- a/sources/platform/proxy/residential_proxy.md +++ b/sources/platform/4-core-concepts/proxy/residential_proxy.md @@ -2,7 +2,7 @@ title: Residential proxy description: Achieve a higher level of anonymity using IP addresses from human users. Access a wider pool of proxies and reduce blocking by websites' anti-scraping measures. sidebar_position: 10.3 -slug: /proxy/residential-proxy +slug: /core-concepts/proxy/residential-proxy --- import Tabs from '@theme/Tabs'; @@ -12,7 +12,7 @@ import TabItem from '@theme/TabItem'; --- -Residential proxies use IP addresses assigned by Internet Service Providers to the homes and offices of actual users. Unlike [datacenter proxies](./datacenter_proxy.md), traffic from residential proxies is indistinguishable from that of legitimate users. +Residential proxies use IP addresses assigned by Internet Service Providers to the homes and offices of actual users. Unlike [datacenter proxies](/platform/core-concepts/proxy/datacenter-proxy), traffic from residential proxies is indistinguishable from that of legitimate users. This solution allows you to access a larger pool of servers than datacenter proxy. This makes it a better option in cases when you need a large number of different IP addresses. @@ -22,7 +22,7 @@ Residential proxies support [IP address rotation](./usage.md#ip-address-rotation ## Connecting to residential proxy -Connecting to residential proxy works the same way as [datacenter proxy](./datacenter_proxy.md), with two differences. +Connecting to residential proxy works the same way as [datacenter proxy](/platform/core-concepts/proxy/datacenter-proxy), with two differences. 1. The `groups` [username parameter](./usage.md#username-parameters) should always specify `RESIDENTIAL`. @@ -30,7 +30,7 @@ Connecting to residential proxy works the same way as [datacenter proxy](./datac ### How to set a proxy group -When using [standard libraries and languages](./datacenter_proxy.md), specify the `groups` parameter in the [username](./usage.md#username-parameters) as `groups-RESIDENTIAL`. +When using [standard libraries and languages](/platform/core-concepts/proxy/datacenter-proxy), specify the `groups` parameter in the [username](/platform/core-concepts/proxy/usage#username-parameters) as `groups-RESIDENTIAL`. For example, your **proxy URL** when using the [got-scraping](https://www.npmjs.com/package/got-scraping) JavaScript library will look like this: @@ -73,7 +73,7 @@ async def main(): ### How to set a proxy country -When using [standard libraries and languages](./datacenter_proxy.md), specify the `country` parameter in the [username](./usage.md#username-parameters) as `country-COUNTRY-CODE`. +When using [standard libraries and languages](/platform/core-concepts/proxy/datacenter-proxy), specify the `country` parameter in the [username](/platform/core-concepts/proxy/usage#username-parameters) as `country-COUNTRY-CODE`. For example, your `username` parameter when using [Python 3](https://docs.python.org/3/) will look like this: @@ -131,11 +131,11 @@ If the proxy server becomes unresponsive or the session expires, a new IP addres > If you really need to persist the same session, you can try sending some data using that session (e.g. every 20 seconds) to keep it alive.
> Providing the connection is not interrupted, this will let you keep the IP address for longer. -To learn more about [sessions](./usage.md#sessions) and [IP address rotation](./usage.md#ip-address-rotation), see the proxy [overview page](./index.md). +To learn more about [sessions](/platform/core-concepts/proxy/usage#sessions) and [IP address rotation](/platform/core-concepts/proxy/usage#ip-address-rotation), see the proxy [overview page](/platform/core-concepts/proxy). ## Tips to keep in mind -[Residential](./index.md) proxies are less predictable than [datacenter](./datacenter_proxy.md) proxies and are priced differently (by number of IPs vs traffic used). Because of this, there are some important things to consider before using residential proxy in your solutions. +[Residential](/platform/core-concepts/proxy) proxies are less predictable than [datacenter](/platform/core-concepts/proxy/datacenter-proxy) proxies and are priced differently (by number of IPs vs traffic used). Because of this, there are some important things to consider before using residential proxy in your solutions. ### Control traffic used by automated browsers {#control-traffic-used-by-automated-browsers} @@ -153,4 +153,4 @@ If your solution requires quickly loaded content, the best option is to set a [s While sessions are persistent, they can be destroyed at any time if the host devices are turned off or disconnected. -For this problem there is no easy solution. One option is to not use residential proxy for larger requests (and use [datacenter](./datacenter_proxy.md) proxy instead). If you have no other choice, expect that interruptions might happen and write your solution with this in mind. +For this problem there is no easy solution. One option is to not use residential proxy for larger requests (and use [datacenter](/platform/core-concepts/proxy/datacenter-proxy) proxy instead). If you have no other choice, expect that interruptions might happen and write your solution with this in mind. diff --git a/sources/platform/proxy/usage.md b/sources/platform/4-core-concepts/proxy/usage.md similarity index 91% rename from sources/platform/proxy/usage.md rename to sources/platform/4-core-concepts/proxy/usage.md index 0203aeb2f4..0182b948c2 100644 --- a/sources/platform/proxy/usage.md +++ b/sources/platform/4-core-concepts/proxy/usage.md @@ -2,7 +2,7 @@ title: Proxy usage description: Learn how to configure and use Apify Proxy. See the required parameters such as the correct username and password. sidebar_position: 10.1 -slug: /proxy/usage +slug: /core-concepts/proxy/usage --- **Learn how to configure and use Apify Proxy. See the required parameters such as the correct username and password.** @@ -49,7 +49,7 @@ http://auto:apify_proxy_EaAFg6CFhc4eKk54Q1HbGDEiUTrk480uZv03@proxy.apify.com:800 If you want to connect to Apify Proxy from Actors running on the Apify Platform, the recommended way is to use built-in proxy configuration tools in the [Apify SDK JavaScript](/sdk/js/docs/guides/proxy-management) or [Apify SDK Python](/sdk/python/docs/concepts/proxy-management) -If you don't want to use these helpers, and want to connect to Apify Proxy manually, you can find the right configuration values in [environment variables](../actors/development/programming_interface/environment_variables.md) provided to the Actor. +If you don't want to use these helpers, and want to connect to Apify Proxy manually, you can find the right configuration values in [environment variables](///platform/building-actors/programming-interface/environment-variables) provided to the Actor. By using this configuration, you ensure that you connect to Apify Proxy directly through the Apify infrastructure, bypassing any external connection via the Internet, thereby improving the connection speed, and ensuring you don't pay for external data transfer. | Parameter | Source / explanation | @@ -70,7 +70,7 @@ const connectionString = `http://auto:${APIFY_PROXY_PASSWORD}@${APIFY_PROXY_HOST The `username` field enables you to pass parameters like **[groups](#proxy-groups)**, **[session ID](#sessions)** and **country** for your proxy connection. -For example, if you're using [datacenter proxies](./datacenter_proxy.md) and want to use the `new_job_123` session using the `SHADER` group, the username will be: +For example, if you're using [datacenter proxies](/platform/core-concepts/proxy/datacenter-proxy) and want to use the `new_job_123` session using the `SHADER` group, the username will be: ```text groups-SHADER,session-new_job_123 @@ -147,12 +147,11 @@ Use [sessions](#sessions) to control how you rotate IP addresses. See our guide Sessions allow you to use the same IP address for multiple connections. In cases where you need to keep the same session (e.g. when you need to log in to a website), it is best to keep the same proxy and so the IP address. On the other hand by switching the IP address, you can avoid being blocked by the website. -To set a new session, pass the `session` parameter in your [username](./usage.md#username-parameters) field when connecting to a proxy. This will serve as the session's ID and an IP address will be assigned to it. To [use that IP address in other requests](/platform/proxy/datacenter-proxy#connecting-to-datacenter-proxies), pass that same session ID in the username field. +To set a new session, pass the `session` parameter in your [username](./usage.md#username-parameters) field when connecting to a proxy. This will serve as the session's ID and an IP address will be assigned to it. To [use that IP address in other requests](/platform/core-concepts/proxy/datacenter-proxy#connecting-to-datacenter-proxies), pass that same session ID in the username field. We recommend you to use [SessionPool](https://crawlee.dev/api/core/class/SessionPool) abstraction when managing sessions. The created session will then store information such as cookies and can be used to generate [browser fingerprints](/academy/anti-scraping/mitigation/generating-fingerprints). You can also assign custom user data such as authorization tokens and specific headers. -Sessions are available for [datacenter](./datacenter_proxy.md) and [residential](./ -residential_proxy.md#session-persistence) proxies. For datacenter proxies, a session persists for -**26 hours** ([more info](./datacenter_proxy.md)). For residential proxies, it persists for **1 minute** ([more info](./residential_proxy.md#session-persistence)) but you can prolong the lifetime by regularly using the session. Google SERP proxies do not support sessions. +Sessions are available for [datacenter](/platform/core-concepts/proxy/datacenter-proxy) and [residential](/platform/core-concepts/proxy/residential-proxy#session-persistence) proxies. For datacenter proxies, a session persists for +**26 hours** ([more info](/platform/core-concepts/proxy/datacenter-proxy)). For residential proxies, it persists for **1 minute** ([more info](/platform/core-concepts/proxy/residential-proxy#session-persistence)) but you can prolong the lifetime by regularly using the session. Google SERP proxies do not support sessions. ## Proxy groups diff --git a/sources/platform/proxy/your_own_proxies.md b/sources/platform/4-core-concepts/proxy/your_own_proxies.md similarity index 87% rename from sources/platform/proxy/your_own_proxies.md rename to sources/platform/4-core-concepts/proxy/your_own_proxies.md index 3f2be86712..a7d4350a79 100644 --- a/sources/platform/proxy/your_own_proxies.md +++ b/sources/platform/4-core-concepts/proxy/your_own_proxies.md @@ -2,7 +2,7 @@ title: Using your own proxies description: Learn how to use your own proxies while using the Apify platform. sidebar_position: 10.5 -slug: /proxy/using-your-own-proxies +slug: /core-concepts/proxy/using-your-own-proxies --- **Learn how to use your own proxies while using the Apify platform.** @@ -15,7 +15,7 @@ In addition to our proxies, you can use your own both in Apify Console and SDK. To use your own proxies with Apify Console, in your Actor's **Input and options** tab, scroll down and open the **Proxy and browser configuration** section. Enter your proxy URLs, and you're good to go. -![Using custom proxy in Apify Console](../images/proxy-custom.png) +![Using custom proxy in Apify Console](../../../platform/images/proxy-custom.png) ## Custom proxies in SDK diff --git a/sources/platform/4-core-concepts/storage/_category_.yml b/sources/platform/4-core-concepts/storage/_category_.yml new file mode 100644 index 0000000000..f1fce84dc5 --- /dev/null +++ b/sources/platform/4-core-concepts/storage/_category_.yml @@ -0,0 +1,2 @@ +label: 'Storage' +position: 2 diff --git a/sources/platform/storage/dataset.md b/sources/platform/4-core-concepts/storage/dataset.md similarity index 86% rename from sources/platform/storage/dataset.md rename to sources/platform/4-core-concepts/storage/dataset.md index b56d6008d9..09b6eea874 100644 --- a/sources/platform/storage/dataset.md +++ b/sources/platform/4-core-concepts/storage/dataset.md @@ -3,7 +3,7 @@ title: Dataset description: Store and export web scraping, crawling or data processing job results. Learn how to access and manage datasets in Apify Console or via API. sidebar_position: 9.2 toc_max_heading_level: 4 -slug: /storage/dataset +slug: /core-concepts/storage/dataset --- **Store and export web scraping, crawling or data processing job results. Learn how to access and manage datasets in Apify Console or via API.** @@ -18,7 +18,7 @@ Dataset storage enables you to sequentially save and retrieve data. A unique dat Typically, datasets comprise results from web scraping, crawling, and data processing jobs. You can visualize this data in a table, where each object is forming a row and its attributes are represented as columns. You have the option to export data in various formats, including JSON, CSV, XML, Excel, HTML Table, RSS or JSONL. > Named datasets are retained indefinitely. -> Unnamed datasets expire after 7 days unless otherwise specified. [Learn more](/platform/storage/usage#named-and-unnamed-storages) +> Unnamed datasets expire after 7 days unless otherwise specified. [Learn more](/platform/core-concepts/storage/usage#named-and-unnamed-storages) Dataset storage is _append-only_ - data can only be added and cannot be modified or deleted once stored. @@ -43,7 +43,7 @@ To view or download a dataset: 2. Select the format & configure other options if desired in **Export dataset** section. 3. Click **Download**. -Utilize the **Actions** menu to modify the dataset's name, which also affects its [retention period](/platform/storage/usage#data-retention), and to adjust [access rights](../collaboration/index.md). The **API** button allows you to explore and test the dataset's [API endpoints](/api/v2/storage-datasets). +Utilize the **Actions** menu to modify the dataset's name, which also affects its [retention period](/platform/core-concepts/storage/usage#data-retention), and to adjust [access rights](/platform/collaboration). The **API** button allows you to explore and test the dataset's [API endpoints](/api/v2/storage-datasets). ![Datasets detail view](./images/datasets-detail.png) @@ -51,9 +51,9 @@ Utilize the **Actions** menu to modify the dataset's name, which also affects it The [Apify API](/api/v2/storage-datasets) enables you programmatic access to your datasets using [HTTP requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods). -If you are accessing your datasets using the `username~store-name` [store ID format](./index.md), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations)tab of **Settings** page of your Apify account. +If you are accessing your datasets using the `username~store-name` [store ID format](/platform/core-concepts/storage), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations)tab of **Settings** page of your Apify account. -> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](../integrations/programming/api.md#authentication)). +> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](//platform/integrations/api#authentication)). To retrieve a list of your datasets, send a GET request to the [Get list of datasets](/api/v2/datasets-get) endpoint. @@ -151,7 +151,7 @@ Check out the [Python API client documentation](/api/client/python/reference/cla #### JavaScript SDK -When working with a JavaScript [Actor](../actors/index.mdx), the [JavaScript SDK](/sdk/js/docs/guides/result-storage#dataset) is an essential tool, especially for dataset management. It simplifies the tasks of storing and retrieving data, seamlessly integrating with the Actor's workflow. Key features of the SDK include the ability to append data, retrieve what is stored, and manage dataset properties effectively. Central to this functionality is the [`Dataset`](/sdk/js/reference/class/Dataset) class. This class allows you to determine where your data is stored - locally or in the Apify cloud. To add data to your chosen datasets, use the [`pushData()`](/sdk/js/reference/class/Dataset#pushData) method. +When working with a JavaScript [Actor](/platform/using-actors), the [JavaScript SDK](/sdk/js/docs/guides/result-storage#dataset) is an essential tool, especially for dataset management. It simplifies the tasks of storing and retrieving data, seamlessly integrating with the Actor's workflow. Key features of the SDK include the ability to append data, retrieve what is stored, and manage dataset properties effectively. Central to this functionality is the [`Dataset`](/sdk/js/reference/class/Dataset) class. This class allows you to determine where your data is stored - locally or in the Apify cloud. To add data to your chosen datasets, use the [`pushData()`](/sdk/js/reference/class/Dataset#pushData) method. Additionally the SDK offers other methods like [`getData()`](/sdk/js/reference/class/Dataset#getData), [`map()`](/sdk/js/reference/class/Dataset#map), and [`reduce()`](/sdk/js/reference/class/Dataset#reduce). For practical applications of these methods, refer to the [example](/sdk/js/docs/examples/map-and-reduce) section. @@ -225,7 +225,7 @@ Check out the [JavaScript SDK documentation](/sdk/js/docs/guides/result-storage# #### Python SDK -For Python [Actors](../actors/index.mdx), the [Python SDK](/sdk/python/docs/concepts/storages#working-with-datasets) is essential. The dataset is represented by a [`Dataset`](/sdk/python/reference/class/Dataset) class. You can use this class to specify whether your data is stored locally or in the Apify cloud and push data to the datasets of your choice using the [`push_data()`](/sdk/python/reference/class/Dataset#push_data) method. For further data manipulation you could also use other methods such as [`get_data()`](/sdk/python/reference/class/Dataset#get_data), [`map()`](/sdk/python/reference/class/Dataset#map) and [`reduce()`](/sdk/python/reference/class/Dataset#reduce). +For Python [Actors](/platform/using-actors), the [Python SDK](/sdk/python/docs/concepts/storages#working-with-datasets) is essential. The dataset is represented by a [`Dataset`](/sdk/python/reference/class/Dataset) class. You can use this class to specify whether your data is stored locally or in the Apify cloud and push data to the datasets of your choice using the [`push_data()`](/sdk/python/reference/class/Dataset#push_data) method. For further data manipulation you could also use other methods such as [`get_data()`](/sdk/python/reference/class/Dataset#get_data), [`map()`](/sdk/python/reference/class/Dataset#map) and [`reduce()`](/sdk/python/reference/class/Dataset#reduce). For datasets stored locally, the data is located at the following path: @@ -380,7 +380,7 @@ By default, the whole result is wrapped in an `` element, while each pag ## Sharing -You can grant [access rights](../collaboration/index.md) to your dataset through the **Share** button under the **Actions** menu. For more details, check the [full list of permissions](../collaboration/list_of_permissions.md). +You can grant [access rights](/platform/collaboration) to your dataset through the **Share** button under the **Actions** menu. For more details, check the [full list of permissions](/platform/collaboration/list-of-permissions). You can also share datasets by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). @@ -388,7 +388,7 @@ For one-off sharing of specific records when access is restricted, you can gener ### Sharing datasets between runs -You can access a dataset from any [Actor](../actors/index.mdx) or [task](../actors/running/tasks.md) run as long as you know its _name_ or _ID_. +You can access a dataset from any [Actor](/platform/using-actors) or [task](///platform/using-actors/tasks) run as long as you know its _name_ or _ID_. To access a dataset from another run using the [Apify SDK](/sdk), open it using the same method as you would with any other dataset. @@ -442,7 +442,7 @@ other_dataset_client = apify_client.dataset('jane-doe/old-dataset') The same applies for the [Apify API](#apify-api) - you can use [the same endpoints](#apify-api) as you would normally do. -See the [Storage overview](/platform/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. +See the [Storage overview](/platform/core-concepts/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. ## Limits diff --git a/sources/platform/storage/images/datasets-app.png b/sources/platform/4-core-concepts/storage/images/datasets-app.png similarity index 100% rename from sources/platform/storage/images/datasets-app.png rename to sources/platform/4-core-concepts/storage/images/datasets-app.png diff --git a/sources/platform/storage/images/datasets-detail.png b/sources/platform/4-core-concepts/storage/images/datasets-detail.png similarity index 100% rename from sources/platform/storage/images/datasets-detail.png rename to sources/platform/4-core-concepts/storage/images/datasets-detail.png diff --git a/sources/platform/storage/images/find-store-id.png b/sources/platform/4-core-concepts/storage/images/find-store-id.png similarity index 100% rename from sources/platform/storage/images/find-store-id.png rename to sources/platform/4-core-concepts/storage/images/find-store-id.png diff --git a/sources/platform/storage/images/key-value-stores-app.png b/sources/platform/4-core-concepts/storage/images/key-value-stores-app.png similarity index 100% rename from sources/platform/storage/images/key-value-stores-app.png rename to sources/platform/4-core-concepts/storage/images/key-value-stores-app.png diff --git a/sources/platform/storage/images/key-value-stores-detail.png b/sources/platform/4-core-concepts/storage/images/key-value-stores-detail.png similarity index 100% rename from sources/platform/storage/images/key-value-stores-detail.png rename to sources/platform/4-core-concepts/storage/images/key-value-stores-detail.png diff --git a/sources/platform/storage/images/overview-api.png b/sources/platform/4-core-concepts/storage/images/overview-api.png similarity index 100% rename from sources/platform/storage/images/overview-api.png rename to sources/platform/4-core-concepts/storage/images/overview-api.png diff --git a/sources/platform/storage/images/request-queue-app.png b/sources/platform/4-core-concepts/storage/images/request-queue-app.png similarity index 100% rename from sources/platform/storage/images/request-queue-app.png rename to sources/platform/4-core-concepts/storage/images/request-queue-app.png diff --git a/sources/platform/storage/images/request-queue-detail.png b/sources/platform/4-core-concepts/storage/images/request-queue-detail.png similarity index 100% rename from sources/platform/storage/images/request-queue-detail.png rename to sources/platform/4-core-concepts/storage/images/request-queue-detail.png diff --git a/sources/platform/storage/key_value_store.md b/sources/platform/4-core-concepts/storage/key_value_store.md similarity index 79% rename from sources/platform/storage/key_value_store.md rename to sources/platform/4-core-concepts/storage/key_value_store.md index 9d86b0aeb7..bca2d1b445 100644 --- a/sources/platform/storage/key_value_store.md +++ b/sources/platform/4-core-concepts/storage/key_value_store.md @@ -3,7 +3,7 @@ title: Key-value store description: Store anything from Actor or task run results, JSON documents, or images. Learn how to access and manage key-value stores from Apify Console or via API. toc_max_heading_level: 4 sidebar_position: 9.3 -slug: /storage/key-value-store +slug: /core-concepts/storage/key-value-store --- **Store anything from Actor or task run results, JSON documents, or images. Learn how to access and manage key-value stores from Apify Console or via API.** @@ -20,7 +20,7 @@ Each Actor run is assigned its own key-value store when it is created. The store Key-value stores are mutable–you can both add entries and delete them. > Named key-value stores are retained indefinitely.
-> Unnamed key-value stores expire after 7 days unless otherwise specified.
> [Learn more](/platform/storage/usage#named-and-unnamed-storages) +> Unnamed key-value stores expire after 7 days unless otherwise specified.
> [Learn more](/platform/core-concepts/storage/usage#named-and-unnamed-storages) ## Basic usage @@ -38,7 +38,7 @@ In [Apify Console](https://console.apify.com), you can view your key-value store ![Key-value stores in app](./images/key-value-stores-app.png) To view a key-value store's content, click on its **Store ID**. -Under the **Actions** menu, you can rename your store (and, in turn extend its [retention period](/platform/storage/usage#named-and-unnamed-storages)) and grant [access rights](../collaboration/index.md) using the **Share** button. +Under the **Actions** menu, you can rename your store (and, in turn extend its [retention period](/platform/core-concepts/storage/usage#named-and-unnamed-storages)) and grant [access rights](/platform/collaboration) using the **Share** button. Click on the **API** button to view and test a store's [API endpoints](/api/v2/storage-key-value-stores). ![Key-value stores detail](./images/key-value-stores-detail.png) @@ -47,9 +47,9 @@ Click on the **API** button to view and test a store's [API endpoints](/api/v2/s The [Apify API](/api/v2/storage-key-value-stores) enables you programmatic access to your key-value stores using [HTTP requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods). -If you are accessing your datasets using the `username~store-name` [store ID format](./index.md), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations) tab of **Settings** page of your Apify account. +If you are accessing your datasets using the `username~store-name` [store ID format](/platform/core-concepts/storage), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations) tab of **Settings** page of your Apify account. -> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](../integrations/programming/api.md#authentication)). +> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](//platform/integrations/api#authentication)). To retrieve a list of your key-value stores, send a GET request to the [Get list of key-value stores](/api/v2/key-value-stores-get) endpoint. @@ -128,11 +128,11 @@ Check out the [Python API client documentation](/api/client/python/reference/cla #### JavaScript SDK -When working with a JavaScript [Actor](../actors/index.mdx), the [JavaScript SDK](/sdk/js/docs/guides/result-storage#key-value-store) is an essential tool, especially for key-value store management. The primary class for this purpose is the [`KeyValueStore`](/sdk/js/reference/class/KeyValueStore). This class allows you to decide whether your data will be stored locally or in the Apify cloud. For data manipulation, it offers the [`getValue()`](/sdk/js/reference/class/KeyValueStore#getValue) and [`setValue()`](/sdk/js/reference/class/KeyValueStore#setValue) methods to retrieve and assign values, respectively. +When working with a JavaScript [Actor](/platform/using-actors), the [JavaScript SDK](/sdk/js/docs/guides/result-storage#key-value-store) is an essential tool, especially for key-value store management. The primary class for this purpose is the [`KeyValueStore`](/sdk/js/reference/class/KeyValueStore). This class allows you to decide whether your data will be stored locally or in the Apify cloud. For data manipulation, it offers the [`getValue()`](/sdk/js/reference/class/KeyValueStore#getValue) and [`setValue()`](/sdk/js/reference/class/KeyValueStore#setValue) methods to retrieve and assign values, respectively. Additionally, you can iterate over the keys in your store using the [`forEachKey()`](/sdk/js/reference/class/KeyValueStore#forEachKey) method. -Every Actor run is linked to a default key-value store that is automatically created for that specific run. If you're running your Actors and opt to store data locally, you can easily supply the [input](../actors/running/input_and_output.md) by placing an _INPUT.json_ file in the corresponding directory of the default key-value store. This method ensures that you Actor has all the necessary data readily available for its execution. +Every Actor run is linked to a default key-value store that is automatically created for that specific run. If you're running your Actors and opt to store data locally, you can easily supply the [input](///platform/using-actors/input-and-output) by placing an _INPUT.json_ file in the corresponding directory of the default key-value store. This method ensures that you Actor has all the necessary data readily available for its execution. You can find _INPUT.json_ and other key-value store files in the location below. @@ -188,15 +188,15 @@ await Actor.setValue('OUTPUT', imageBuffer, { contentType: 'image/jpeg' }); await Actor.exit(); ``` -The `Actor.getInput()` method is not only a shortcut to `Actor.getValue('INPUT')`; it is also compatible with [`Actor.metamorph()`](../actors/development/programming_interface/metamorph.md). This is because a metamorphed Actor run's input is stored in the _INPUT-METAMORPH-1_ key instead of _INPUT_, which hosts the original input. +The `Actor.getInput()` method is not only a shortcut to `Actor.getValue('INPUT')`; it is also compatible with [`Actor.metamorph()`](/platform/building-actors/programming-interface/metamorph). This is because a metamorphed Actor run's input is stored in the _INPUT-METAMORPH-1_ key instead of _INPUT_, which hosts the original input. Check out the [JavaScript SDK documentation](/sdk/js/docs/guides/result-storage#key-value-store) and the `KeyValueStore` class's [API reference](/sdk/js/reference/class/KeyValueStore) for details on managing your key-value stores with the JavaScript SDK. #### Python SDK -For Python [Actor](../actors/index.mdx), the [Python SDK](/sdk/python/docs/concepts/storages#working-with-key-value-stores) is essential. The key-value store is represented by a [`KeyValueStore`](/sdk/python/reference/class/KeyValueStore) class. You can use this class to specify whether your data is stored locally or in the Apify cloud. For further data manipulation it offers [`get_value()`](/sdk/python/reference/class/KeyValueStore#get_value) and [`set_value()`](/sdk/python/reference/class/KeyValueStore#set_value) methods to retrieve and assign values, respectively. +For Python [Actor](/platform/using-actors), the [Python SDK](/sdk/python/docs/concepts/storages#working-with-key-value-stores) is essential. The key-value store is represented by a [`KeyValueStore`](/sdk/python/reference/class/KeyValueStore) class. You can use this class to specify whether your data is stored locally or in the Apify cloud. For further data manipulation it offers [`get_value()`](/sdk/python/reference/class/KeyValueStore#get_value) and [`set_value()`](/sdk/python/reference/class/KeyValueStore#set_value) methods to retrieve and assign values, respectively. -Every Actor run is linked to a default key-value store that is automatically created for that specific run. If you're running your Actors and opt to store data locally, you can easily supply the [input](../actors/running/input_and_output.md) by placing an _INPUT.json_ file in the corresponding directory of the default key-value store. This method ensures that you Actor has all the necessary data readily available for its execution. +Every Actor run is linked to a default key-value store that is automatically created for that specific run. If you're running your Actors and opt to store data locally, you can easily supply the [input](///platform/using-actors/input-and-output) by placing an _INPUT.json_ file in the corresponding directory of the default key-value store. This method ensures that you Actor has all the necessary data readily available for its execution. You can find _INPUT.json_ and other key-value store files in the location below. @@ -240,7 +240,7 @@ async def main(): await Actor.set_value(key='OUTPUT', value=image_buffer, content_type='image/jpeg') ``` -The `Actor.get_input()` method is not only a shortcut to `Actor.get_value('INPUT')`; it is also compatible with [`Actor.metamorph()`](../actors/development/programming_interface/metamorph.md). This is because a metamorphed Actor run's input is stored in the _INPUT-METAMORPH-1_ key instead of _INPUT_, which hosts the original input. +The `Actor.get_input()` method is not only a shortcut to `Actor.get_value('INPUT')`; it is also compatible with [`Actor.metamorph()`](/platform/building-actors/programming-interface/metamorph). This is because a metamorphed Actor run's input is stored in the _INPUT-METAMORPH-1_ key instead of _INPUT_, which hosts the original input. Check out the [Python SDK documentation](/sdk/python/docs/concepts/storages#working-with-key-value-stores) and the `KeyValueStore` class's [API reference](/sdk/python/reference/class/KeyValueStore) for details on managing your key-value stores with the Python SDK. @@ -254,7 +254,7 @@ _Using the [JavaScript SDK](/sdk/js/reference/class/KeyValueStore#setValue) or o ## Sharing -You can grant [access rights](../collaboration/index.md) to your key-value store through the **Share** button under the **Actions** menu. For more details check the [full list of permissions](../collaboration/list_of_permissions.md). +You can grant [access rights](/platform/collaboration) to your key-value store through the **Share** button under the **Actions** menu. For more details check the [full list of permissions](/platform/collaboration/list-of-permissions). You can also share key-value stores by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). @@ -262,7 +262,7 @@ For one-off sharing of specific records when access is restricted, you can gener ### Sharing key-value stores between runs -You can access a key-value store from any [Actor](../actors/index.mdx) or [task](../actors/running/tasks.md) run as long as you know its _name_ or _ID_. +You can access a key-value store from any [Actor](/platform/using-actors) or [task](///platform/using-actors/tasks) run as long as you know its _name_ or _ID_. To access a key-value store from another run using the [Apify SDK](/sdk), open it using the same method as you would do with any other store. @@ -316,7 +316,7 @@ other_store_client = apify_client.key_value_store('jane-doe/old-store') The same applies for the [Apify API](#apify-api) - you can use [the same endpoints](#apify-api) as you would normally do. -Check out the [Storage overview](/platform/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. +Check out the [Storage overview](/platform/core-concepts/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. ## Data consistency diff --git a/sources/platform/storage/request_queue.md b/sources/platform/4-core-concepts/storage/request_queue.md similarity index 90% rename from sources/platform/storage/request_queue.md rename to sources/platform/4-core-concepts/storage/request_queue.md index 29e3d71cd3..f73aedf2da 100644 --- a/sources/platform/storage/request_queue.md +++ b/sources/platform/4-core-concepts/storage/request_queue.md @@ -3,7 +3,7 @@ title: Request queue description: Queue URLs for an Actor to visit in its run. Learn how to share your queues between Actor runs. Access and manage request queues from Apify Console or via API. toc_max_heading_level: 4 sidebar_position: 9.4 -slug: /storage/request-queue +slug: /core-concepts/storage/request-queue --- **Queue URLs for an Actor to visit in its run. Learn how to share your queues between Actor runs. Access and manage request queues from Apify Console or via API.** @@ -18,7 +18,7 @@ Request queues enable you to enqueue and retrieve requests such as URLs with an The storage system for request queues accommodates both breadth-first and depth-first crawling strategies, along with the inclusion of custom data attributes. This system enables you to check if certain URLs have already been encountered, add new URLs to the queue, and retrieve the next set of URLs for processing. > Named request queues are retained indefinitely.
-> Unnamed request queues expire after 7 days unless otherwise specified.
> [Learn more](/platform/storage/usage#named-and-unnamed-storages) +> Unnamed request queues expire after 7 days unless otherwise specified.
> [Learn more](/platform/core-concepts/storage/usage#named-and-unnamed-storages) ## Basic usage @@ -37,7 +37,7 @@ In the [Apify Console](https://console.apify.com), you can view your request que To view a request queue, click on its **Queue ID**. Under the **Actions** menu, you can rename your queue's name (and, in turn, its -[retention period](/platform/storage/usage#named-and-unnamed-storages)) and [access rights](../collaboration/index.md) using the **Share** button. +[retention period](/platform/core-concepts/storage/usage#named-and-unnamed-storages)) and [access rights](/platform/collaboration) using the **Share** button. Click on the **API** button to view and test a queue's [API endpoints](/api/v2/storage-request-queues). ![Request queues detail](./images/request-queue-detail.png) @@ -46,9 +46,9 @@ Click on the **API** button to view and test a queue's [API endpoints](/api/v2/s The [Apify API](/api/v2/storage-request-queues) allows you programmatic access to your request queues using [HTTP requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods). -If you are accessing your datasets using the `username~store-name` [store ID format](./index.md), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations) page of your Apify account. +If you are accessing your datasets using the `username~store-name` [store ID format](/platform/core-concepts/storage), you will need to use your secret API token. You can find the token (and your user ID) on the [Integrations](https://console.apify.com/account#/integrations) page of your Apify account. -> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](../integrations/programming/api.md#authentication)). +> When providing your API authentication token, we recommend using the request's `Authorization` header, rather than the URL. ([More info](//platform/integrations/api#authentication)). To get a list of your request queues, send a GET request to the [Get list of request queues](/api/v2/request-queues-get) endpoint. @@ -141,9 +141,9 @@ Check out the [Python API client documentation](/api/client/python/reference/cla #### JavaScript SDK -When working with a JavaScript [Actor](../actors/index.mdx), the [JavaScript SDK](/sdk/js/docs/guides/request-storage#request-queue) is an essential tool, especially for request queue management. The primary class for this purpose is the [`RequestQueue`](/sdk/js/reference/class/RequestQueue) class. Use this class to decide whether your data is stored locally or in the Apify cloud. +When working with a JavaScript [Actor](/platform/using-actors), the [JavaScript SDK](/sdk/js/docs/guides/request-storage#request-queue) is an essential tool, especially for request queue management. The primary class for this purpose is the [`RequestQueue`](/sdk/js/reference/class/RequestQueue) class. Use this class to decide whether your data is stored locally or in the Apify cloud. -If you are building a JavaScript [Actor](../actors/index.mdx), you will be using the [JavaScript SDK](/sdk/js/docs/guides/request-storage#request-queue). The request queue is represented by a [`RequestQueue`](/sdk/js/reference/class/RequestQueue) class. You can use the class to specify whether your data is stored locally or in the Apify cloud and [enqueue new URLs](/sdk/js/reference/class/RequestQueue#addRequests). +If you are building a JavaScript [Actor](/platform/using-actors), you will be using the [JavaScript SDK](/sdk/js/docs/guides/request-storage#request-queue). The request queue is represented by a [`RequestQueue`](/sdk/js/reference/class/RequestQueue) class. You can use the class to specify whether your data is stored locally or in the Apify cloud and [enqueue new URLs](/sdk/js/reference/class/RequestQueue#addRequests). Every Actor run is automatically linked with a default request queue, initiated upon adding the first request. This queue is primarily utilized for storing URLs to be crawled during the particular Actor run, though its use is not mandatory. For enhanced flexibility, you can establish named queues. These named queues offer the advantage of being shareable across different Actors or various Actor runs, facilitating a more interconnected and efficient process. @@ -214,7 +214,7 @@ Check out the [JavaScript SDK documentation](/sdk/js/docs/guides/request-storage #### Python SDK -For Python [Actor](../actors/index.mdx) development, the [Python SDK](/sdk/python/docs/concepts/storages#working-with-request-queues) the in essential. The request queue is represented by [`RequestQueue`](/sdk/python/reference/class/RequestQueue) class. Utilize this class to determine whether your data is stored locally or in the Apify cloud. For managing your data, it provides the capability to [enqueue new URLs](/sdk/python/reference/class/RequestQueue#add_requests), facilitating seamless integration and operation within your Actor. +For Python [Actor](/platform/using-actors) development, the [Python SDK](/sdk/python/docs/concepts/storages#working-with-request-queues) the in essential. The request queue is represented by [`RequestQueue`](/sdk/python/reference/class/RequestQueue) class. Utilize this class to determine whether your data is stored locally or in the Apify cloud. For managing your data, it provides the capability to [enqueue new URLs](/sdk/python/reference/class/RequestQueue#add_requests), facilitating seamless integration and operation within your Actor. Every Actor run is automatically connected to a default request queue, established specifically for that run upon the addition of the first request. If you're operating your Actors and choose to utilize this queue, it typically serves to store URLs for crawling in the respective Actor run, though its use is not mandatory. To extend functionality, you have the option to create named queue, which offer the flexibility to be shared among different Actors or across multiple Actor runs. @@ -556,7 +556,7 @@ A detailed tutorial on how to process one request queue with multiple Actor runs ## Sharing -You can grant [access rights](../collaboration/index.md) to your request queue through the **Share** button under the **Actions** menu. For more details check the [full list of permissions](../collaboration/list_of_permissions.md). +You can grant [access rights](/platform/collaboration) to your request queue through the **Share** button under the **Actions** menu. For more details check the [full list of permissions](/platform/collaboration/list-of-permissions). You can also share request queues by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). @@ -564,7 +564,7 @@ For one-off sharing of specific records when access is restricted, you can gener ### Sharing request queues between runs -You can access a request queue from any [Actor](../actors/index.mdx) or [task](../actors/running/tasks.md) run as long as you know its _name_ or _ID_. +You can access a request queue from any [Actor](/platform/using-actors) or [task](///platform/using-actors/tasks) run as long as you know its _name_ or _ID_. To access a request queue from another run using the [Apify SDK](/sdk), open it using the same method like you would do with any other request queue. @@ -618,7 +618,7 @@ other_queue_client = apify_client.request_queue('jane-doe/old-queue') The same applies for the [Apify API](#apify-api) - you can use [the same endpoints](#apify-api) as you would normally do. -Check out the [Storage overview](/platform/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. +Check out the [Storage overview](/platform/core-concepts/storage/usage#sharing-storages-between-runs) for details on sharing storages between runs. ## Limits diff --git a/sources/platform/storage/usage.md b/sources/platform/4-core-concepts/storage/usage.md similarity index 84% rename from sources/platform/storage/usage.md rename to sources/platform/4-core-concepts/storage/usage.md index ac55550620..c5125ab33b 100644 --- a/sources/platform/storage/usage.md +++ b/sources/platform/4-core-concepts/storage/usage.md @@ -3,7 +3,7 @@ title: Storage usage description: Learn how to effectively use Apify's storage options. Understand key aspects of data retention, rate limiting, and secure sharing. sidebar_position: 9.1 category: platform -slug: /storage/usage +slug: /core-concepts/storage/usage --- import StoragePricingCalculator from "@site/src/components/StoragePricingCalculator"; @@ -14,22 +14,22 @@ import StoragePricingCalculator from "@site/src/components/StoragePricingCalcula ## Dataset -[Dataset](./dataset.md) storage allows you to store a series of data objects, such as results from web scraping, crawling, or data processing jobs. You can export your datasets in JSON, CSV, XML, RSS, Excel, or HTML formats. +[Dataset](/platform/core-concepts/storage/dataset) storage allows you to store a series of data objects, such as results from web scraping, crawling, or data processing jobs. You can export your datasets in JSON, CSV, XML, RSS, Excel, or HTML formats. -![Dataset graphic](../images/datasets-overview.png) +![Dataset graphic](../../images/datasets-overview.png) ## Key-value store -The [key-value store](./key_value_store.md) is ideal for saving data records such as files, screenshots of web pages, and PDFs or for persisting your Actor's state. The records are accessible under a unique name and can be written and read quickly. +The [key-value store](/platform/core-concepts/storage/key-value-store) is ideal for saving data records such as files, screenshots of web pages, and PDFs or for persisting your Actor's state. The records are accessible under a unique name and can be written and read quickly. -![Key-value store graphic](../images/key-value-overview.svg) +![Key-value store graphic](../../images/key-value-overview.svg) ## Request queue -[Request queues](./request_queue.md) allow you to dynamically maintain a queue of URLs of web pages. You can use this when recursively crawling websites: you start from initial URLs and add new links as they are found while skipping duplicates. +[Request queues](/platform/core-concepts/storage/request-queue) allow you to dynamically maintain a queue of URLs of web pages. You can use this when recursively crawling websites: you start from initial URLs and add new links as they are found while skipping duplicates. -![Request queue graphic](../images/request-queue-overview.svg) +![Request queue graphic](../../images/request-queue-overview.svg) ## Basic usage @@ -160,7 +160,7 @@ For example, storage names `janedoe~my-storage-1` and `janedoe~web-scrape-result ## Sharing -You can grant [access rights](../collaboration/index.md) to other Apify users to view or modify your storages. Check the [full list of permissions](../collaboration/list_of_permissions.md). +You can grant [access rights](/platform/collaboration) to other Apify users to view or modify your storages. Check the [full list of permissions](/platform/collaboration/list-of-permissions). You can also share storages by link using their ID or name, depending on your account or resource-level general access setting. Learn how link-based access works in [General resource access](/platform/collaboration/general-resource-access). @@ -174,11 +174,11 @@ If your storage resource is set to _restricted_, all API calls must include a va ### Sharing storages between runs -Storage can be accessed from any [Actor](../actors/index.mdx) or [task](../actors/running/tasks.md) run, provided you have its _name_ or _ID_. You can access and manage storages from other runs using the same methods or endpoints as with storages from your current run. +Storage can be accessed from any [Actor](/platform/using-actors) or [task](///platform/using-actors/tasks) run, provided you have its _name_ or _ID_. You can access and manage storages from other runs using the same methods or endpoints as with storages from your current run. -[Datasets](./dataset.md) and [key-value stores](./key_value_store.md) support concurrent use by multiple Actors. Thus, several Actors or tasks can simultaneously write data to a single dataset or key-value store. Similarly, multiple runs can read data from datasets and key-value stores at the same time. +[Datasets](/platform/core-concepts/storage/dataset) and [key-value stores](/platform/core-concepts/storage/key-value-store) support concurrent use by multiple Actors. Thus, several Actors or tasks can simultaneously write data to a single dataset or key-value store. Similarly, multiple runs can read data from datasets and key-value stores at the same time. -[Request queues](./request_queue.md), on the other hand, only allow multiple runs to add new data. A request queue can only be processed by one Actor or task run at any one time. +[Request queues](/platform/core-concepts/storage/request-queue), on the other hand, only allow multiple runs to add new data. A request queue can only be processed by one Actor or task run at any one time. > When multiple runs try to write data to a storage simultaneously, the order of data writing cannot be controlled. Data is written as each request is processed.
> Similar principle applies in key-value stores and request queues, when a delete request for a record precedes a read request for the same record, the read request will fail. diff --git a/sources/platform/5-building-actors/_category_.yml b/sources/platform/5-building-actors/_category_.yml new file mode 100644 index 0000000000..cfbb868ec4 --- /dev/null +++ b/sources/platform/5-building-actors/_category_.yml @@ -0,0 +1,2 @@ +label: 'Building Actors' +position: 5 \ No newline at end of file diff --git a/sources/platform/actors/development/actor_definition/actor_json.md b/sources/platform/5-building-actors/actor_definition/actor_json.md similarity index 81% rename from sources/platform/actors/development/actor_definition/actor_json.md rename to sources/platform/5-building-actors/actor_definition/actor_json.md index 8023a1fce0..2f9800a3d5 100644 --- a/sources/platform/actors/development/actor_definition/actor_json.md +++ b/sources/platform/5-building-actors/actor_definition/actor_json.md @@ -1,7 +1,7 @@ --- title: actor.json description: Learn how to write the main Actor config in the `.actor/actor.json` file. -slug: /actors/development/actor-definition/actor-json +slug: /building-actors/actor-definition/actor-json sidebar_position: 1 --- @@ -69,15 +69,15 @@ Actor `name`, `version`, `buildTag`, and `environmentVariables` are currently on | `actorSpecification` | Required | The version of the Actor specification. This property must be set to `1`, which is the only version available. | | `name` | Required | The name of the Actor. | | `version` | Required | The version of the Actor, specified in the format `[Number].[Number]`, e.g., `0.1`, `0.3`, `1.0`, `1.3`, etc. | -| `buildTag` | Optional | The tag name to be applied to a successful build of the Actor. If not specified, defaults to `latest`. Refer to the [builds](../builds_and_runs/builds.md) for more information. | +| `buildTag` | Optional | The tag name to be applied to a successful build of the Actor. If not specified, defaults to `latest`. Refer to the [builds](///platform/building-actors/builds-and-runs/builds) for more information. | | `environmentVariables` | Optional | A map of environment variables to be used during local development. These variables will also be applied to the Actor when deployed on the Apify platform. For more details, see the [environment variables](/cli/docs/vars) section of Apify CLI documentation. | -| `dockerfile` | Optional | The path to the Dockerfile to be used for building the Actor on the platform. If not specified, the system will search for Dockerfiles in the `.actor/Dockerfile` and `Dockerfile` paths, in that order. Refer to the [Dockerfile](./docker.md) section for more information. | +| `dockerfile` | Optional | The path to the Dockerfile to be used for building the Actor on the platform. If not specified, the system will search for Dockerfiles in the `.actor/Dockerfile` and `Dockerfile` paths, in that order. Refer to the [Dockerfile](/platform/building-actors/actor-definition/dockerfile) section for more information. | | `dockerContextDir` | Optional | The path to the directory to be used as the Docker context when building the Actor. The path is relative to the location of the `actor.json` file. This property is useful for monorepos containing multiple Actors. Refer to the [Actor monorepos](../deployment/source_types.md#actor-monorepos) section for more details. | | `readme` | Optional | The path to the README file to be used on the platform. If not specified, the system will look for README files in the `.actor/README.md` and `README.md` paths, in that order of preference. Check out [Apify Marketing Playbook to learn how to write a quality README files](https://apify.notion.site/How-to-create-an-Actor-README-759a1614daa54bee834ee39fe4d98bc2) guidance. | -| `input` | Optional | You can embed your [input schema](./input_schema/index.md) object directly in `actor.json` under the `input` field. You can also provide a path to a custom input schema. If not provided, the input schema at `.actor/INPUT_SCHEMA.json` or `INPUT_SCHEMA.json` is used, in this order of preference. | +| `input` | Optional | You can embed your [input schema](/platform/using-actors/development/actor-definition/input-schema) object directly in `actor.json` under the `input` field. You can also provide a path to a custom input schema. If not provided, the input schema at `.actor/INPUT_SCHEMA.json` or `INPUT_SCHEMA.json` is used, in this order of preference. | | `changelog` | Optional | The path to the CHANGELOG file displayed in the Information tab of the Actor in Apify Console next to Readme. If not provided, the CHANGELOG at `.actor/CHANGELOG.md` or `CHANGELOG.md` is used, in this order of preference. Your Actor doesn't need to have a CHANGELOG but it is a good practice to keep it updated for published Actors. | -| `storages.dataset` | Optional | You can define the schema of the items in your dataset under the `storages.dataset` field. This can be either an embedded object or a path to a JSON schema file. [Read more](/platform/actors/development/actor-definition/dataset-schema) about Actor dataset schemas. | -| `minMemoryMbytes` | Optional | Specifies the minimum amount of memory in megabytes required by the Actor to run. Requires an _integer_ value. If both `minMemoryMbytes` and `maxMemoryMbytes` are set, then `minMemoryMbytes` must be equal or lower than `maxMemoryMbytes`. Refer to the [Usage and resources](https://docs.apify.com/platform/actors/running/usage-and-resources#memory) for more details about memory allocation. | -| `maxMemoryMbytes` | Optional | Specifies the maximum amount of memory in megabytes required by the Actor to run. It can be used to control the costs of run, especially when developing pay per result Actors. Requires an _integer_ value. Refer to the [Usage and resources](https://docs.apify.com/platform/actors/running/usage-and-resources#memory) for more details about memory allocation. | -| `usesStandbyMode` | Optional | Boolean specifying whether the Actor will have [Standby mode](../programming_interface/actor_standby.md) enabled. | +| `storages.dataset` | Optional | You can define the schema of the items in your dataset under the `storages.dataset` field. This can be either an embedded object or a path to a JSON schema file. [Read more](/platform/using-actors/development/actor-definition/dataset-schema) about Actor dataset schemas. | +| `minMemoryMbytes` | Optional | Specifies the minimum amount of memory in megabytes required by the Actor to run. Requires an _integer_ value. If both `minMemoryMbytes` and `maxMemoryMbytes` are set, then `minMemoryMbytes` must be equal or lower than `maxMemoryMbytes`. Refer to the [Usage and resources](https://docs.apify.com/platform/using-actors/running/usage-and-resources#memory) for more details about memory allocation. | +| `maxMemoryMbytes` | Optional | Specifies the maximum amount of memory in megabytes required by the Actor to run. It can be used to control the costs of run, especially when developing pay per result Actors. Requires an _integer_ value. Refer to the [Usage and resources](https://docs.apify.com/platform/using-actors/running/usage-and-resources#memory) for more details about memory allocation. | +| `usesStandbyMode` | Optional | Boolean specifying whether the Actor will have [Standby mode](/platform/building-actors/programming-interface/standby) enabled. | | `webServerSchema` | Optional | Defines an OpenAPI v3 schema for the web server running in the Actor. This can be either an embedded object or a path to a JSON schema file. Use this when your Actor starts its own HTTP server and you want to describe its interface. | diff --git a/sources/platform/actors/development/actor_definition/dataset_schema/index.md b/sources/platform/5-building-actors/actor_definition/dataset_schema/index.md similarity index 100% rename from sources/platform/actors/development/actor_definition/dataset_schema/index.md rename to sources/platform/5-building-actors/actor_definition/dataset_schema/index.md diff --git a/sources/platform/actors/development/actor_definition/dataset_schema/validation.md b/sources/platform/5-building-actors/actor_definition/dataset_schema/validation.md similarity index 96% rename from sources/platform/actors/development/actor_definition/dataset_schema/validation.md rename to sources/platform/5-building-actors/actor_definition/dataset_schema/validation.md index e96c9def67..25ff7c2d61 100644 --- a/sources/platform/actors/development/actor_definition/dataset_schema/validation.md +++ b/sources/platform/5-building-actors/actor_definition/dataset_schema/validation.md @@ -1,7 +1,7 @@ --- title: Dataset validation description: Specify the dataset schema within the Actors so you can add monitoring and validation at the field level. -slug: /actors/development/actor-definition/dataset-schema/validation +slug: /building-actors/actor-definition/dataset-schema/validation --- import Tabs from '@theme/Tabs'; @@ -104,7 +104,7 @@ If the data you attempt to store in the dataset is _invalid_ (meaning any of the } ``` -The type of the AJV validation error object is [here](https://github.com/ajv-validator/ajv/blob/master/lib/types/index.ts#L86). +The AJV validation error object type is defined in the [AJV TypeScript type definitions](https://github.com/ajv-validator/ajv/blob/master/lib/types/index.ts#L86). If you use the Apify JS client or Apify SDK and call `pushData` function you can access the validation errors in a `try catch` block like this: diff --git a/sources/platform/actors/development/actor_definition/docker.md b/sources/platform/5-building-actors/actor_definition/docker.md similarity index 95% rename from sources/platform/actors/development/actor_definition/docker.md rename to sources/platform/5-building-actors/actor_definition/docker.md index c3ccc6a776..19bcef7134 100644 --- a/sources/platform/actors/development/actor_definition/docker.md +++ b/sources/platform/5-building-actors/actor_definition/docker.md @@ -1,7 +1,7 @@ --- title: Dockerfile description: Learn about the available Docker images you can use as a base for your Apify Actors. Choose the right base image based on your Actor's requirements and the programming language you're using. -slug: /actors/development/actor-definition/dockerfile +slug: /building-actors/actor-definition/dockerfile sidebar_position: 4 --- @@ -9,7 +9,7 @@ sidebar_position: 4 --- -When developing an [Actor](/sources/platform/actors/index.mdx) on the Apify platform, you can choose from a variety of pre-built Docker images to serve as the base for your Actor. These base images come with pre-installed dependencies and tools, making it easier to set up your development environment and ensuring consistent behavior across different environments. +When developing an [Actor](/platform/using-actors) on the Apify platform, you can choose from a variety of pre-built Docker images to serve as the base for your Actor. These base images come with pre-installed dependencies and tools, making it easier to set up your development environment and ensuring consistent behavior across different environments. ## Base Docker images @@ -109,7 +109,7 @@ This means the system expects the source code to be in `main.js` by default. If :::tip Optimization tips -You can check out various optimization tips for Dockerfile in our [Performance](../performance.md) documentation. +You can check out various optimization tips for Dockerfile in our [Performance](/platform/building-actors/performance) documentation. ::: diff --git a/sources/platform/actors/development/actor_definition/images/output-schema-example.png b/sources/platform/5-building-actors/actor_definition/images/output-schema-example.png similarity index 100% rename from sources/platform/actors/development/actor_definition/images/output-schema-example.png rename to sources/platform/5-building-actors/actor_definition/images/output-schema-example.png diff --git a/sources/platform/actors/development/actor_definition/index.md b/sources/platform/5-building-actors/actor_definition/index.md similarity index 54% rename from sources/platform/actors/development/actor_definition/index.md rename to sources/platform/5-building-actors/actor_definition/index.md index 00bfcd895f..39331cc5b9 100644 --- a/sources/platform/actors/development/actor_definition/index.md +++ b/sources/platform/5-building-actors/actor_definition/index.md @@ -15,16 +15,16 @@ Basically, Actors are programs packaged as [Docker images](https://hub.docker.co Actors have the following elements: -- The main **[actor.json](./actor_json.md)** file contains **metadata** such as the Actor name, description, author, version, and links pointing to the other definition files below. -- **[Dockerfile](./docker.md)** which specifies where is the Actor's source code, how to build it, and run it. +- The main **[actor.json](/platform/building-actors/actor-definition/actor-json)** file contains **metadata** such as the Actor name, description, author, version, and links pointing to the other definition files below. +- **[Dockerfile](/platform/building-actors/actor-definition/dockerfile)** which specifies where is the Actor's source code, how to build it, and run it. - **Documentation** in the form of a **README.md** file. -- **[Input](./input_schema/index.md)** and **[dataset schemas](/platform/actors/development/actor-definition/dataset-schema)** that describe what input the Actor requires and what results it produces. -- Access to an out-of-box **[storage](../../../storage/index.md)** system for Actor data, results, and files. +- **[Input](/platform/using-actors/development/actor-definition/input-schema)** and **[dataset schemas](/platform/using-actors/development/actor-definition/dataset-schema)** that describe what input the Actor requires and what results it produces. +- Access to an out-of-box **[storage](/platform/core-concepts/storage)** system for Actor data, results, and files. The documentation and the input/dataset schemas make it possible for people to easily understand what the Actor does, enter the required inputs both in the user interface or API, and integrate the Actor's results with their other workflows. Actors can easily call and interact with each other, enabling building more complex systems on top of simple ones. -The Apify platform provides an open [API](/api/v2), cron-style [scheduler](../../schedules), [webhooks](../../../integrations/programming/webhooks/index.md), and [integrations](../../integrations) to services such as Zapier or Make, which make it easy for users to integrate Actors with their existing workflows. Anyone is welcome to [publish Actors](/platform/actors/publishing) in [Apify Store](https://apify.com/store), and you can even [monetize your Actors](/platform/actors/publishing/monetize). +The Apify platform provides an open [API](/api/v2), cron-style [scheduler](../../schedules), [webhooks](/platform/integrations/webhooks), and [integrations](../../integrations) to services such as Zapier or Make, which make it easy for users to integrate Actors with their existing workflows. Anyone is welcome to [publish Actors](/platform/publishing-and-monetization) in [Apify Store](https://apify.com/store), and you can even [monetize your Actors](/platform/publishing-and-monetization/monetize). -Actors can be developed and run locally and then easily deployed to the Apify platform using the [Apify CLI](/cli) or a [GitHub integration](../../../integrations/programming/github.md). For more details, see the [Deployment](../deployment/index.md) section. +Actors can be developed and run locally and then easily deployed to the Apify platform using the [Apify CLI](/cli) or a [GitHub integration](/platform/integrations/github). For more details, see the [Deployment](/platform/using-actors/development/deployment) section. > **To get a better idea of what Apify Actors are, visit [Apify Store](https://apify.com/store), and try out some of them!** diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-colors.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-colors.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-colors.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-colors.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-country.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-country.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-country.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-country.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-dataset-json.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-dataset-json.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-dataset-json.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-dataset-json.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-dataset.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-dataset.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-dataset.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-dataset.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-datasets.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-datasets.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-datasets.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-datasets.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-absolute.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-absolute.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-absolute.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-absolute.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-both.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-both.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-both.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-both.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-relative.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-relative.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-date-relative.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-date-relative.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-example.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-example.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-example.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-example.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-fileupload-input.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-fileupload-input.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-fileupload-input.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-fileupload-input.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-fileupload-modal.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-fileupload-modal.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-fileupload-modal.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-fileupload-modal.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-memory.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-memory.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-memory.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-memory.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-options.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-options.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-options.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-options.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-page-function.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-page-function.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-page-function.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-page-function.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-proxy.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-proxy.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-proxy.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-proxy.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-start-urls.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-start-urls.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-start-urls.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-start-urls.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-schema-user.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-user.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-schema-user.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-schema-user.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/input-ui-website-content-crawler.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/input-ui-website-content-crawler.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/input-ui-website-content-crawler.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/input-ui-website-content-crawler.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/images/secret-input-editor.png b/sources/platform/5-building-actors/actor_definition/input_schema/images/secret-input-editor.png similarity index 100% rename from sources/platform/actors/development/actor_definition/input_schema/images/secret-input-editor.png rename to sources/platform/5-building-actors/actor_definition/input_schema/images/secret-input-editor.png diff --git a/sources/platform/actors/development/actor_definition/input_schema/index.md b/sources/platform/5-building-actors/actor_definition/input_schema/index.md similarity index 93% rename from sources/platform/actors/development/actor_definition/input_schema/index.md rename to sources/platform/5-building-actors/actor_definition/input_schema/index.md index 3ede079356..bc7d3c7378 100644 --- a/sources/platform/actors/development/actor_definition/input_schema/index.md +++ b/sources/platform/5-building-actors/actor_definition/input_schema/index.md @@ -78,4 +78,4 @@ The actual input object passed from the autogenerated input UI to the Actor then } ``` -Next, let's take a look at [input schema specification](./specification.md), and the possibility of using input schema to enable users to pass [secrets](./secret_input.md). +Next, let's take a look at [input schema specification](/platform/building-actors/actor-definition/input-schema/specification/v1), and the possibility of using input schema to enable users to pass [secrets](///platform/building-actors/actor-definition/input-schema/secret-input). diff --git a/sources/platform/actors/development/actor_definition/input_schema/secret_input.md b/sources/platform/5-building-actors/actor_definition/input_schema/secret_input.md similarity index 95% rename from sources/platform/actors/development/actor_definition/input_schema/secret_input.md rename to sources/platform/5-building-actors/actor_definition/input_schema/secret_input.md index 061cfd48e2..155b84e4ba 100644 --- a/sources/platform/actors/development/actor_definition/input_schema/secret_input.md +++ b/sources/platform/5-building-actors/actor_definition/input_schema/secret_input.md @@ -1,7 +1,7 @@ --- title: Secret input description: Learn about making some Actor input fields secret and encrypted. Ideal for passing passwords, API tokens, or login cookies to Actors. -slug: /actors/development/actor-definition/input-schema/secret-input +slug: /building-actors/actor-definition/input-schema/secret-input --- **Learn about making some Actor input fields secret and encrypted. Ideal for passing passwords, API tokens, or login cookies to Actors.** @@ -12,7 +12,7 @@ The secret input feature lets you mark specific input fields of an Actor as sens ## How to set a secret input field -To make an input field secret, you need to add a `"isSecret": true` setting to the input field in the Actor's [input schema](./index.md), like this: +To make an input field secret, you need to add a `"isSecret": true` setting to the input field in the Actor's [input schema](/platform/using-actors/development/actor-definition/input-schema), like this: ```json diff --git a/sources/platform/actors/development/actor_definition/input_schema/specification.md b/sources/platform/5-building-actors/actor_definition/input_schema/specification.md similarity index 99% rename from sources/platform/actors/development/actor_definition/input_schema/specification.md rename to sources/platform/5-building-actors/actor_definition/input_schema/specification.md index 6939af0c39..c6cd3375e3 100644 --- a/sources/platform/actors/development/actor_definition/input_schema/specification.md +++ b/sources/platform/5-building-actors/actor_definition/input_schema/specification.md @@ -2,7 +2,7 @@ title: Actor input schema specification sidebar_position: 2 description: Learn how to define and validate a schema for your Actor's input with code examples. Provide an autogenerated input UI for your Actor's users. -slug: /actors/development/actor-definition/input-schema/specification/v1 +slug: /building-actors/actor-definition/input-schema/specification/v1 toc_max_heading_level: 5 sidebar_label: Input schema specification --- diff --git a/sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-run.png b/sources/platform/5-building-actors/actor_definition/key_value_store_schema/images/kv-store-schema-example-run.png similarity index 100% rename from sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-run.png rename to sources/platform/5-building-actors/actor_definition/key_value_store_schema/images/kv-store-schema-example-run.png diff --git a/sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-storage.png b/sources/platform/5-building-actors/actor_definition/key_value_store_schema/images/kv-store-schema-example-storage.png similarity index 100% rename from sources/platform/actors/development/actor_definition/key_value_store_schema/images/kv-store-schema-example-storage.png rename to sources/platform/5-building-actors/actor_definition/key_value_store_schema/images/kv-store-schema-example-storage.png diff --git a/sources/platform/actors/development/actor_definition/key_value_store_schema/index.md b/sources/platform/5-building-actors/actor_definition/key_value_store_schema/index.md similarity index 100% rename from sources/platform/actors/development/actor_definition/key_value_store_schema/index.md rename to sources/platform/5-building-actors/actor_definition/key_value_store_schema/index.md diff --git a/sources/platform/actors/development/actor_definition/source_code.md b/sources/platform/5-building-actors/actor_definition/source_code.md similarity index 92% rename from sources/platform/actors/development/actor_definition/source_code.md rename to sources/platform/5-building-actors/actor_definition/source_code.md index cb91041fc9..7f8f95ba0f 100644 --- a/sources/platform/actors/development/actor_definition/source_code.md +++ b/sources/platform/5-building-actors/actor_definition/source_code.md @@ -1,7 +1,7 @@ --- title: Source code description: Learn about the Actor's source code placement and its structure. -slug: /actors/development/actor-definition/source-code +slug: /building-actors/actor-definition/source-code sidebar_position: 4 --- @@ -9,7 +9,7 @@ sidebar_position: 4 --- -The Apify Actor's source code placement is defined by its [Dockerfile](./docker.md). If you have created the Actor from one of Apify's [templates](https://apify.com/templates) then it's by convention placed in the `/src` directory. +The Apify Actor's source code placement is defined by its [Dockerfile](/platform/building-actors/actor-definition/dockerfile). If you have created the Actor from one of Apify's [templates](https://apify.com/templates) then it's by convention placed in the `/src` directory. You have the flexibility to choose any programming language, technologies, and dependencies (such as Chrome browser, Selenium, Cypress, or others) for your projects. The only requirement is to define a Dockerfile that builds the image for your Actor, including all dependencies and your source code. diff --git a/sources/platform/actors/development/automated_tests.md b/sources/platform/5-building-actors/automated_tests.md similarity index 99% rename from sources/platform/actors/development/automated_tests.md rename to sources/platform/5-building-actors/automated_tests.md index 967d3f4e80..c9026cbd0b 100644 --- a/sources/platform/actors/development/automated_tests.md +++ b/sources/platform/5-building-actors/automated_tests.md @@ -1,7 +1,7 @@ --- title: Automated tests for Actors description: Learn how to automate ongoing testing and make sure your Actors perform over time. See code examples for configuring the Actor Testing Actor. -slug: /actors/development/automated-tests +slug: /building-actors/automated-tests sidebar_position: 9 sidebar_label: Automated tests --- diff --git a/sources/platform/actors/development/builds_and_runs/builds.md b/sources/platform/5-building-actors/builds_and_runs/builds.md similarity index 84% rename from sources/platform/actors/development/builds_and_runs/builds.md rename to sources/platform/5-building-actors/builds_and_runs/builds.md index 873aa31515..ad82fc262f 100644 --- a/sources/platform/actors/development/builds_and_runs/builds.md +++ b/sources/platform/5-building-actors/builds_and_runs/builds.md @@ -2,7 +2,7 @@ title: Builds description: Learn about Actor build numbers, versioning, and how to use specific Actor version in runs. Understand an Actor's lifecycle and manage its cache. sidebar_position: 7 -slug: /actors/development/builds-and-runs/builds +slug: /building-actors/builds-and-runs/builds --- **Learn about Actor build numbers, versioning, and how to use specific Actor version in runs. Understand an Actor's lifecycle and manage its cache.** @@ -11,7 +11,7 @@ slug: /actors/development/builds-and-runs/builds ## Understand Actor builds -Before an Actor can be run, it needs to be built. The build process creates a snapshot of a specific version of the Actor's settings, including its [source code](../actor_definition/source_code.md) and [environment variables](../programming_interface/environment_variables.md). This snapshot is then used to create a Docker image containing everything the Actor needs for its run, such as `npm` packages, web browsers, etc. +Before an Actor can be run, it needs to be built. The build process creates a snapshot of a specific version of the Actor's settings, including its [source code](/platform/building-actors/actor-definition/source-code) and [environment variables](///platform/building-actors/programming-interface/environment-variables). This snapshot is then used to create a Docker image containing everything the Actor needs for its run, such as `npm` packages, web browsers, etc. ### Build numbers @@ -27,7 +27,7 @@ By default, builds have the following resource allocations: - Timeout: _1800_ seconds - Memory: `4096 MB` -Check out the [Resource limits](../../running/index.md) section for more details. +Check out the [Resource limits](///platform/using-actors/usage-and-resources) section for more details. ## Versioning diff --git a/sources/platform/actors/development/builds_and_runs/index.md b/sources/platform/5-building-actors/builds_and_runs/index.md similarity index 100% rename from sources/platform/actors/development/builds_and_runs/index.md rename to sources/platform/5-building-actors/builds_and_runs/index.md diff --git a/sources/platform/actors/development/builds_and_runs/runs.md b/sources/platform/5-building-actors/builds_and_runs/runs.md similarity index 66% rename from sources/platform/actors/development/builds_and_runs/runs.md rename to sources/platform/5-building-actors/builds_and_runs/runs.md index 19f9567a2a..3264b02739 100644 --- a/sources/platform/actors/development/builds_and_runs/runs.md +++ b/sources/platform/5-building-actors/builds_and_runs/runs.md @@ -2,7 +2,7 @@ title: Runs sidebar_position: 8 description: Learn about Actor runs, how to start them, and how to manage them. -slug: /actors/development/builds-and-runs/runs +slug: /building-actors/builds-and-runs/runs --- **Learn about Actor runs, how to start them, and how to manage them.** @@ -17,12 +17,12 @@ You can start an Actor in several ways: - Manually from the [Apify Console](https://console.apify.com/actors) UI - Via the [Apify API](/api/v2/act-runs-post) -- Using the [Scheduler](../../../schedules.md) provided by the Apify platform -- By one of the available [integrations](../../../integrations/index.mdx) +- Using the [Scheduler](/platform/using-actors/schedules) provided by the Apify platform +- By one of the available [integrations](/platform/integrations) ## Input and environment variables -The run receives input via the `INPUT` record of its default [key-value store](../../../storage/key_value_store.md). Environment variables are also passed to the run. For more information about environment variables check the [Environment variables](../programming_interface/environment_variables.md) section. +The run receives input via the `INPUT` record of its default [key-value store](/platform/core-concepts/storage/key-value-store). Environment variables are also passed to the run. For more information about environment variables check the [Environment variables](///platform/building-actors/programming-interface/environment-variables) section. ## Run duration and timeout diff --git a/sources/platform/actors/development/builds_and_runs/state_persistence.md b/sources/platform/5-building-actors/builds_and_runs/state_persistence.md similarity index 93% rename from sources/platform/actors/development/builds_and_runs/state_persistence.md rename to sources/platform/5-building-actors/builds_and_runs/state_persistence.md index 218b88f387..5962730442 100644 --- a/sources/platform/actors/development/builds_and_runs/state_persistence.md +++ b/sources/platform/5-building-actors/builds_and_runs/state_persistence.md @@ -1,7 +1,7 @@ --- title: State persistence description: Learn how to maintain an Actor's state to prevent data loss during unexpected restarts. Includes code examples for handling server migrations. -slug: /actors/development/builds-and-runs/state-persistence +slug: /building-actors/builds-and-runs/state-persistence --- **Learn how to maintain an Actor's state to prevent data loss during unexpected restarts. Includes code examples for handling server migrations.** @@ -11,7 +11,7 @@ import TabItem from '@theme/TabItem'; --- -Long-running [Actor](../../index.mdx) jobs may need to migrate between servers. Without state persistence, your job's progress is lost during migration, causing it to restart from the beginning on the new server. This can be costly and time-consuming. +Long-running [Actor](/platform) jobs may need to migrate between servers. Without state persistence, your job's progress is lost during migration, causing it to restart from the beginning on the new server. This can be costly and time-consuming. To prevent data loss, long-running Actors should: diff --git a/sources/platform/actors/development/deployment/continuous_integration.md b/sources/platform/5-building-actors/deployment/continuous_integration.md similarity index 99% rename from sources/platform/actors/development/deployment/continuous_integration.md rename to sources/platform/5-building-actors/deployment/continuous_integration.md index 9f7d0e3bc7..0a86691da8 100644 --- a/sources/platform/actors/development/deployment/continuous_integration.md +++ b/sources/platform/5-building-actors/deployment/continuous_integration.md @@ -2,7 +2,7 @@ title: Continuous integration for Actors sidebar_label: Continuous integration description: Learn how to integrate your Actors by setting up automated builds, deploys, and testing for your Actors. -slug: /actors/development/deployment/continuous-integration +slug: /building-actors/deployment/continuous-integration sidebar_position: 2 --- diff --git a/sources/platform/actors/development/deployment/images/actor-source-types.png b/sources/platform/5-building-actors/deployment/images/actor-source-types.png similarity index 100% rename from sources/platform/actors/development/deployment/images/actor-source-types.png rename to sources/platform/5-building-actors/deployment/images/actor-source-types.png diff --git a/sources/platform/actors/development/deployment/images/actor-source.png b/sources/platform/5-building-actors/deployment/images/actor-source.png similarity index 100% rename from sources/platform/actors/development/deployment/images/actor-source.png rename to sources/platform/5-building-actors/deployment/images/actor-source.png diff --git a/sources/platform/actors/development/deployment/images/ci-add-build-url.png b/sources/platform/5-building-actors/deployment/images/ci-add-build-url.png similarity index 100% rename from sources/platform/actors/development/deployment/images/ci-add-build-url.png rename to sources/platform/5-building-actors/deployment/images/ci-add-build-url.png diff --git a/sources/platform/actors/development/deployment/images/ci-add-token.png b/sources/platform/5-building-actors/deployment/images/ci-add-token.png similarity index 100% rename from sources/platform/actors/development/deployment/images/ci-add-token.png rename to sources/platform/5-building-actors/deployment/images/ci-add-token.png diff --git a/sources/platform/actors/development/deployment/images/ci-builds.png b/sources/platform/5-building-actors/deployment/images/ci-builds.png similarity index 100% rename from sources/platform/actors/development/deployment/images/ci-builds.png rename to sources/platform/5-building-actors/deployment/images/ci-builds.png diff --git a/sources/platform/actors/development/deployment/images/ci-github-integration.png b/sources/platform/5-building-actors/deployment/images/ci-github-integration.png similarity index 100% rename from sources/platform/actors/development/deployment/images/ci-github-integration.png rename to sources/platform/5-building-actors/deployment/images/ci-github-integration.png diff --git a/sources/platform/actors/development/deployment/images/ci-token.png b/sources/platform/5-building-actors/deployment/images/ci-token.png similarity index 100% rename from sources/platform/actors/development/deployment/images/ci-token.png rename to sources/platform/5-building-actors/deployment/images/ci-token.png diff --git a/sources/platform/actors/development/deployment/images/deployment-keys.png b/sources/platform/5-building-actors/deployment/images/deployment-keys.png similarity index 100% rename from sources/platform/actors/development/deployment/images/deployment-keys.png rename to sources/platform/5-building-actors/deployment/images/deployment-keys.png diff --git a/sources/platform/actors/development/deployment/images/source-types-gist-settings.png b/sources/platform/5-building-actors/deployment/images/source-types-gist-settings.png old mode 100755 new mode 100644 similarity index 100% rename from sources/platform/actors/development/deployment/images/source-types-gist-settings.png rename to sources/platform/5-building-actors/deployment/images/source-types-gist-settings.png diff --git a/sources/platform/actors/development/deployment/images/testing-tasks.png b/sources/platform/5-building-actors/deployment/images/testing-tasks.png similarity index 100% rename from sources/platform/actors/development/deployment/images/testing-tasks.png rename to sources/platform/5-building-actors/deployment/images/testing-tasks.png diff --git a/sources/platform/actors/development/deployment/index.md b/sources/platform/5-building-actors/deployment/index.md similarity index 91% rename from sources/platform/actors/development/deployment/index.md rename to sources/platform/5-building-actors/deployment/index.md index ccbd537a3e..4fd81ce80c 100644 --- a/sources/platform/actors/development/deployment/index.md +++ b/sources/platform/5-building-actors/deployment/index.md @@ -9,7 +9,7 @@ slug: /actors/development/deployment --- -Deploying an Actor involves uploading your [source code](/platform/actors/development/actor-definition) and [building](/platform/actors/development/builds-and-runs/builds) it on the Apify platform. Once deployed, you can run and scale your Actor in the cloud. +Deploying an Actor involves uploading your [source code](/platform/using-actors/development/actor-definition) and [building](/platform/using-actors/development/builds-and-runs/builds) it on the Apify platform. Once deployed, you can run and scale your Actor in the cloud. ## Deploy using Apify CLI diff --git a/sources/platform/actors/development/deployment/source_types.md b/sources/platform/5-building-actors/deployment/source_types.md similarity index 87% rename from sources/platform/actors/development/deployment/source_types.md rename to sources/platform/5-building-actors/deployment/source_types.md index f747fcc628..2ffa39aa26 100644 --- a/sources/platform/actors/development/deployment/source_types.md +++ b/sources/platform/5-building-actors/deployment/source_types.md @@ -1,7 +1,7 @@ --- title: Source types description: Learn about Apify Actor source types and how to deploy an Actor from GitHub using CLI or Gist. -slug: /actors/development/deployment/source-types +slug: /building-actors/deployment/source-types sidebar_position: 1 --- @@ -21,11 +21,11 @@ This section explains the various sources types available for Apify Actors and h ## Web IDE -This is the default option when your Actor's source code is hosted on the Apify platform. It offers quick previews and updates to your source code, easy file and directory browsing, and direct testing of the [`INPUT_SCHEMA.json`](/platform/actors/development/actor-definition/input-schema) on the Apify platform. +This is the default option when your Actor's source code is hosted on the Apify platform. It offers quick previews and updates to your source code, easy file and directory browsing, and direct testing of the [`INPUT_SCHEMA.json`](/platform/using-actors/development/actor-definition/input-schema) on the Apify platform. -A `Dockerfile` is mandatory for all Actors. When using the default NodeJS Dockerfile, you'll typically need `main.js` for your source code and `package.json` for [NPM](https://www.npmjs.com/) package configurations. +A `Dockerfile` is mandatory for all Actors. When using the default NodeJS Dockerfile, you'll typically need `main.js` for your source code and `package.json` for [npm](https://www.npmjs.com/) package configurations. -For more information on creating custom Dockerfiles or using Apify's base images, refer to the [Dockerfile](/platform/actors/development/actor-definition/dockerfile#custom-dockerfile) and [base Docker images](/platform/actors/development/actor-definition/dockerfile#base-docker-images) documentation. +For more information on creating custom Dockerfiles or using Apify's base images, refer to the [Dockerfile](/platform/using-actors/development/actor-definition/dockerfile#custom-dockerfile) and [base Docker images](/platform/using-actors/development/actor-definition/dockerfile#base-docker-images) documentation. ## Git repository @@ -73,7 +73,7 @@ Remember that each key can only be used once per Git hosting service (GitHub, Bi ### Actor monorepos -To manage multiple Actors in a single repository, use the `dockerContextDix` property in the [Actor definition](/platform/actors/development/actor-definition/actor-json) to set the Docker context directory (if not provided then the repository root is used). In the Dockerfile, copy both the Actor's source and any shared code into the Docker image. +To manage multiple Actors in a single repository, use the `dockerContextDix` property in the [Actor definition](/platform/using-actors/development/actor-definition/actor-json) to set the Docker context directory (if not provided then the repository root is used). In the Dockerfile, copy both the Actor's source and any shared code into the Docker image. To enable sharing Dockerfiles between multiple Actors, the Actor build process passes the `ACTOR_PATH_IN_DOCKER_CONTEXT` build argument to the Docker build. It contains the relative path from `dockerContextDir` to the directory selected as the root of the Actor in the Apify Console (the "directory" part of the Actor's git URL). diff --git a/sources/platform/5-building-actors/index.md b/sources/platform/5-building-actors/index.md new file mode 100644 index 0000000000..76114b20e7 --- /dev/null +++ b/sources/platform/5-building-actors/index.md @@ -0,0 +1,51 @@ +--- +title: "Building Actors" +description: "Create custom Actors" +sidebar_position: 1 +slug: /building-actors +--- + +**Read about the technical part of building Apify Actors. Learn to define Actor inputs, build new versions, persist Actor state, and choose base Docker images.** + +--- + +This section will guide you through the whole story of [Actor](/platform) development. + +You can follow chapters sequentially from [Getting started](/platform/getting-started), where you learn how to create your first Actor in just a few minutes, through the more technical sections describing the whole Actor model, up to the [Performance](/platform/building-actors/performance) section, where you learn how to fine-tune your Actor to get the most out of the Apify platform. + +import Card from "@site/src/components/Card"; +import CardGrid from "@site/src/components/CardGrid"; + + + + + + + + + +--- + +If your Actor allows for integration with external services, or performs some post-processing of datasets generated by other Actors, check out the section [integration-ready Actors](/platform/integrations/actors/integration-ready-actors). + +After your development, you can jump to the section [Publishing and monetization](/platform/publishing-and-monetization) to learn about how to publish your Actor in [Apify Store](https://apify.com/store) and monetize it by renting it out to users of the platform. diff --git a/sources/platform/actors/development/performance.md b/sources/platform/5-building-actors/performance.md similarity index 94% rename from sources/platform/actors/development/performance.md rename to sources/platform/5-building-actors/performance.md index 70d70666d1..fd22a4d319 100644 --- a/sources/platform/actors/development/performance.md +++ b/sources/platform/5-building-actors/performance.md @@ -2,7 +2,7 @@ title: Performance sidebar_position: 9 description: Learn how to get the maximum value out of your Actors, minimize costs, and maximize results. -slug: /actors/development/performance +slug: /building-actors/performance --- **Learn how to get the maximum value out of your Actors, minimize costs, and maximize results.** @@ -56,4 +56,4 @@ We first copy the `package.json`, `package-lock.json` files , and install the de ### Use standardized images to accelerate Actor startup times -Using one of [Apify's standardized images](https://github.com/apify/apify-actor-docker), can accelerate the Actor startup time. These images are cached on each worker machine, so only the layers you added in your Actor's [Dockerfile](./actor_definition/docker.md) need to be pulled. +Using one of [Apify's standardized images](https://github.com/apify/apify-actor-docker), can accelerate the Actor startup time. These images are cached on each worker machine, so only the layers you added in your Actor's [Dockerfile](/platform/building-actors/actor-definition/dockerfile) need to be pulled. diff --git a/sources/platform/actors/development/programming_interface/actor_standby.md b/sources/platform/5-building-actors/programming_interface/actor_standby.md similarity index 99% rename from sources/platform/actors/development/programming_interface/actor_standby.md rename to sources/platform/5-building-actors/programming_interface/actor_standby.md index 0b6d3cd0a7..aae7187f43 100644 --- a/sources/platform/actors/development/programming_interface/actor_standby.md +++ b/sources/platform/5-building-actors/programming_interface/actor_standby.md @@ -1,7 +1,7 @@ --- title: Standby mode description: Use the Actor as a real-time API server. -slug: /actors/development/programming-interface/standby +slug: /building-actors/programming-interface/standby sidebar_position: 9 --- diff --git a/sources/platform/actors/development/programming_interface/basic_commands.md b/sources/platform/5-building-actors/programming_interface/basic_commands.md similarity index 97% rename from sources/platform/actors/development/programming_interface/basic_commands.md rename to sources/platform/5-building-actors/programming_interface/basic_commands.md index 105009ecf4..1874559ebb 100644 --- a/sources/platform/actors/development/programming_interface/basic_commands.md +++ b/sources/platform/5-building-actors/programming_interface/basic_commands.md @@ -1,7 +1,7 @@ --- title: Basic commands description: Learn how to use basic commands of the Apify SDK for both JavaScript and Python. -slug: /actors/development/programming-interface/basic-commands +slug: /building-actors/programming-interface/basic-commands sidebar_position: 2 --- @@ -100,7 +100,7 @@ Usually, the file is called `INPUT`, but the exact key is defined in the `ACTOR_ ## Key-value store access -Use the [Key-value store](../../../storage/key_value_store.md) to read and write arbitrary files +Use the [Key-value store](/platform/core-concepts/storage/key-value-store) to read and write arbitrary files @@ -150,7 +150,7 @@ async def main(): ## Push results to the dataset -Store larger results in a [Dataset](../../../storage/dataset.md), an append-only object storage +Store larger results in a [Dataset](/platform/core-concepts/storage/dataset), an append-only object storage Note that Datasets can optionally be equipped with the schema that ensures only certain kinds of objects are stored in them. diff --git a/sources/platform/actors/development/programming_interface/container_web_server.md b/sources/platform/5-building-actors/programming_interface/container_web_server.md similarity index 91% rename from sources/platform/actors/development/programming_interface/container_web_server.md rename to sources/platform/5-building-actors/programming_interface/container_web_server.md index aff8b31dfd..5f9abf4415 100644 --- a/sources/platform/actors/development/programming_interface/container_web_server.md +++ b/sources/platform/5-building-actors/programming_interface/container_web_server.md @@ -1,7 +1,7 @@ --- title: Container web server description: Learn about how to run a web server inside your Actor, which enables you to communicate with the outer world via both UI and API. -slug: /actors/development/programming-interface/container-web-server +slug: /building-actors/programming-interface/container-web-server sidebar_position: 7 --- @@ -16,7 +16,7 @@ Each Actor run is assigned a unique URL (e.g. `kmdo7wpzlshygi.runs.apify.net`) t :::tip Using Actors as an API -The container web server provides a way how to connect to one specific Actor run. To enable using your Actor as an API, with a pre-defined hostname, load balancing and autoscaling, check out [Actor Standby](./actor_standby.md). +The container web server provides a way how to connect to one specific Actor run. To enable using your Actor as an API, with a pre-defined hostname, load balancing and autoscaling, check out [Actor Standby](/platform/building-actors/programming-interface/standby). ::: @@ -36,7 +36,7 @@ The web server inside the container must listen on the port specified by the `AC 1. Define the `ACTOR_WEB_SERVER_PORT` environment variable with your desired port number. -Check out [Custom environment variables](./environment_variables.md) for more details. +Check out [Custom environment variables](///platform/building-actors/programming-interface/environment-variables) for more details. ## Example: Start a simple web server diff --git a/sources/platform/actors/development/programming_interface/environment_variables.md b/sources/platform/5-building-actors/programming_interface/environment_variables.md similarity index 91% rename from sources/platform/actors/development/programming_interface/environment_variables.md rename to sources/platform/5-building-actors/programming_interface/environment_variables.md index b64da0dc06..095b4f2de4 100644 --- a/sources/platform/actors/development/programming_interface/environment_variables.md +++ b/sources/platform/5-building-actors/programming_interface/environment_variables.md @@ -1,7 +1,7 @@ --- title: Actor environment variables description: Learn how to provide your Actor with context that determines its behavior through a plethora of pre-defined environment variables offered by the Apify SDK. -slug: /actors/development/programming-interface/environment-variables +slug: /building-actors/programming-interface/environment-variables sidebar_position: 3 sidebar_label: Environment variables --- @@ -43,11 +43,11 @@ Here's a table of key system environment variables: | `ACTOR_BUILD_NUMBER` | Build number of the Actor build used in the run. | | `ACTOR_BUILD_TAGS` | A comma-separated list of tags of the Actor build used in the run. Note that this environment variable is assigned at the time of start of the Actor and doesn't change over time, even if the assigned build tags change. | | `ACTOR_TASK_ID` | ID of the Actor task. Empty if Actor is run outside of any task, e.g. directly using the API. | -| `ACTOR_EVENTS_WEBSOCKET_URL` | Websocket URL where Actor may listen for [events](/platform/actors/development/programming-interface/system-events) from Actor platform. | +| `ACTOR_EVENTS_WEBSOCKET_URL` | Websocket URL where Actor may listen for [events](/platform/using-actors/development/programming-interface/system-events) from Actor platform. | | `ACTOR_DEFAULT_DATASET_ID` | Unique identifier for the default dataset associated with the current Actor run. | | `ACTOR_DEFAULT_KEY_VALUE_STORE_ID` | Unique identifier for the default key-value store associated with the current Actor run. | | `ACTOR_DEFAULT_REQUEST_QUEUE_ID` | Unique identifier for the default request queue associated with the current Actor run. | -| `ACTOR_INPUT_KEY` | Key of the record in the default key-value store that holds the [Actor input](/platform/actors/running/input-and-output#input). | +| `ACTOR_INPUT_KEY` | Key of the record in the default key-value store that holds the [Actor input](/platform/using-actors/running/input-and-output#input). | | `ACTOR_MAX_PAID_DATASET_ITEMS` | For paid-per-result Actors, the user-set limit on returned results. Do not exceed this limit. | | `ACTOR_MAX_TOTAL_CHARGE_USD` | For pay-per-event Actors, the user-set limit on run cost. Do not exceed this limit. | | `APIFY_HEADLESS` | If **1**, web browsers inside the Actor should run in headless mode (no windowing system available). | @@ -56,19 +56,19 @@ Here's a table of key system environment variables: | `APIFY_PROXY_PASSWORD` | Password for accessing Apify Proxy services. This password enables the Actor to utilize proxy servers on behalf of the user who initiated the Actor run. | | `APIFY_PROXY_PORT` | TCP port number to be used for connecting to the Apify Proxy. | | `APIFY_PROXY_STATUS_URL` | URL for retrieving proxy status information. Appending `?format=json` to this URL returns the data in JSON format for programmatic processing. | -| `ACTOR_STANDBY_URL` | URL for accessing web servers of Actor runs in the [Actor Standby](/platform/actors/development/programming-interface/standby) mode. | +| `ACTOR_STANDBY_URL` | URL for accessing web servers of Actor runs in the [Actor Standby](/platform/using-actors/development/programming-interface/standby) mode. | | `ACTOR_STARTED_AT` | Date when the Actor was started. | | `ACTOR_TIMEOUT_AT` | Date when the Actor will time out. | | `APIFY_TOKEN` | API token of the user who started the Actor. | | `APIFY_USER_ID` | ID of the user who started the Actor. May differ from the Actor owner. | | `APIFY_USER_IS_PAYING` | If it is `1`, it means that the user who started the Actor is a paying user. | -| `ACTOR_WEB_SERVER_PORT` | TCP port for the Actor to start an HTTP server on. This server can be used to receive external messages or expose monitoring and control interfaces. The server also receives messages from the [Actor Standby](/platform/actors/development/programming-interface/standby) mode. | +| `ACTOR_WEB_SERVER_PORT` | TCP port for the Actor to start an HTTP server on. This server can be used to receive external messages or expose monitoring and control interfaces. The server also receives messages from the [Actor Standby](/platform/using-actors/development/programming-interface/standby) mode. | | `ACTOR_WEB_SERVER_URL` | Unique public URL for accessing the Actor run web server from the outside world. | | `APIFY_API_PUBLIC_BASE_URL` | Public URL of the Apify API. May be used to interact with the platform programmatically. Typically set to `api.apify.com`. | | `APIFY_DEDICATED_CPUS` | Number of CPU cores reserved for the actor, based on allocated memory. | | `APIFY_WORKFLOW_KEY` | Identifier used for grouping related runs and API calls together. | -| `APIFY_META_ORIGIN` | Specifies how an Actor run was started. Possible values are in [Runs and builds](/platform/actors/running/runs-and-builds#origin) documentation. | -| `APIFY_INPUT_SECRETS_KEY_FILE` | Path to the secret key used to decrypt [Secret inputs](/platform/actors/development/actor-definition/input-schema/secret-input). | +| `APIFY_META_ORIGIN` | Specifies how an Actor run was started. Possible values are in [Runs and builds](/platform/using-actors/running/runs-and-builds#origin) documentation. | +| `APIFY_INPUT_SECRETS_KEY_FILE` | Path to the secret key used to decrypt [Secret inputs](/platform/using-actors/development/actor-definition/input-schema/secret-input). | | `APIFY_INPUT_SECRETS_KEY_PASSPHRASE` | Passphrase for the input secret key specified in `APIFY_INPUT_SECRETS_KEY_FILE`. | @@ -119,7 +119,7 @@ For sensitive data like API keys or passwords, enable the **Secret** option. Thi Once you start a build, you cannot change its environment variables. To use different variables, you must create a new build. -Learn more in [Builds](../builds_and_runs/builds.md). +Learn more in [Builds](///platform/building-actors/builds-and-runs/builds). ::: ## Access environment variables diff --git a/sources/platform/actors/development/programming_interface/images/environment-vatiables-source.png b/sources/platform/5-building-actors/programming_interface/images/environment-vatiables-source.png old mode 100755 new mode 100644 similarity index 100% rename from sources/platform/actors/development/programming_interface/images/environment-vatiables-source.png rename to sources/platform/5-building-actors/programming_interface/images/environment-vatiables-source.png diff --git a/sources/platform/actors/development/programming_interface/images/standby-creators.png b/sources/platform/5-building-actors/programming_interface/images/standby-creators.png similarity index 100% rename from sources/platform/actors/development/programming_interface/images/standby-creators.png rename to sources/platform/5-building-actors/programming_interface/images/standby-creators.png diff --git a/sources/platform/actors/development/programming_interface/images/status-message.png b/sources/platform/5-building-actors/programming_interface/images/status-message.png similarity index 100% rename from sources/platform/actors/development/programming_interface/images/status-message.png rename to sources/platform/5-building-actors/programming_interface/images/status-message.png diff --git a/sources/platform/actors/development/programming_interface/index.mdx b/sources/platform/5-building-actors/programming_interface/index.mdx similarity index 77% rename from sources/platform/actors/development/programming_interface/index.mdx rename to sources/platform/5-building-actors/programming_interface/index.mdx index e1a9c3d484..b62ec67dd5 100644 --- a/sources/platform/actors/development/programming_interface/index.mdx +++ b/sources/platform/5-building-actors/programming_interface/index.mdx @@ -2,7 +2,7 @@ title: Programming interface sidebar_position: 4 description: Learn about the programming interface of Apify Actors, important commands and features provided by the Apify SDK, and how to use them in your Actors. -slug: /actors/development/programming-interface +slug: /building-actors/programming-interface --- **Learn about the programming interface of Apify Actors, important commands and features provided by the Apify SDK, and how to use them in your Actors.** @@ -18,36 +18,36 @@ This chapter will guide you through all the commands you need to build your firs diff --git a/sources/platform/actors/development/programming_interface/metamorph.md b/sources/platform/5-building-actors/programming_interface/metamorph.md similarity index 98% rename from sources/platform/actors/development/programming_interface/metamorph.md rename to sources/platform/5-building-actors/programming_interface/metamorph.md index 26c2fd3d65..59e5ac5116 100644 --- a/sources/platform/actors/development/programming_interface/metamorph.md +++ b/sources/platform/5-building-actors/programming_interface/metamorph.md @@ -1,7 +1,7 @@ --- title: Metamorph description: The metamorph operation transforms an Actor run into the run of another Actor with a new input. -slug: /actors/development/programming-interface/metamorph +slug: /building-actors/programming-interface/metamorph sidebar_position: 8 --- diff --git a/sources/platform/actors/development/programming_interface/status_messages.md b/sources/platform/5-building-actors/programming_interface/status_messages.md similarity index 97% rename from sources/platform/actors/development/programming_interface/status_messages.md rename to sources/platform/5-building-actors/programming_interface/status_messages.md index 832b8f9188..06da7a48ea 100644 --- a/sources/platform/actors/development/programming_interface/status_messages.md +++ b/sources/platform/5-building-actors/programming_interface/status_messages.md @@ -1,7 +1,7 @@ --- title: Status messages description: Learn how to use custom status messages to inform users about the progress of an Actor. -slug: /actors/development/programming-interface/status-messages +slug: /building-actors/programming-interface/status-messages sidebar_position: 3 --- diff --git a/sources/platform/actors/development/programming_interface/system_events.md b/sources/platform/5-building-actors/programming_interface/system_events.md similarity index 98% rename from sources/platform/actors/development/programming_interface/system_events.md rename to sources/platform/5-building-actors/programming_interface/system_events.md index bcb9e8c10c..d0cdfbb0dd 100644 --- a/sources/platform/actors/development/programming_interface/system_events.md +++ b/sources/platform/5-building-actors/programming_interface/system_events.md @@ -1,7 +1,7 @@ --- title: System events in Apify Actors description: Learn about system events sent to your Actor and how to benefit from them. -slug: /actors/development/programming-interface/system-events +slug: /building-actors/programming-interface/system-events sidebar_position: 4 sidebar_label: System events --- diff --git a/sources/platform/6-publishing-and-monetization/_category_.yml b/sources/platform/6-publishing-and-monetization/_category_.yml new file mode 100644 index 0000000000..cfd3645d53 --- /dev/null +++ b/sources/platform/6-publishing-and-monetization/_category_.yml @@ -0,0 +1,2 @@ +label: 'Publishing & monetization' +position: 6 \ No newline at end of file diff --git a/sources/platform/actors/publishing/badge.mdx b/sources/platform/6-publishing-and-monetization/badge.mdx similarity index 94% rename from sources/platform/actors/publishing/badge.mdx rename to sources/platform/6-publishing-and-monetization/badge.mdx index e3eef7599b..f96c24b97c 100644 --- a/sources/platform/actors/publishing/badge.mdx +++ b/sources/platform/6-publishing-and-monetization/badge.mdx @@ -1,7 +1,7 @@ --- title: Actor status badge description: The Actor status badge can be embedded in the README or documentation to show users the current status and usage of your Actor on the Apify platform. -slug: /actors/publishing/status-badge +slug: /publishing-and-monetization/status-badge sidebar_position: 4 --- @@ -49,7 +49,7 @@ In order to embed the badge in the HTML documentation, just use it as an image w ### Supported Actor states -The badge indicates the state of the Actor in the Apify platform as the result of the [automated testing](../development/automated_tests.md). +The badge indicates the state of the Actor in the Apify platform as the result of the [automated testing](/platform/building-actors/automated-tests). #### Actor OK diff --git a/sources/platform/actors/publishing/images/actor-SEO.webp b/sources/platform/6-publishing-and-monetization/images/actor-SEO.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-SEO.webp rename to sources/platform/6-publishing-and-monetization/images/actor-SEO.webp diff --git a/sources/platform/actors/publishing/images/actor-analytisc.png b/sources/platform/6-publishing-and-monetization/images/actor-analytisc.png similarity index 100% rename from sources/platform/actors/publishing/images/actor-analytisc.png rename to sources/platform/6-publishing-and-monetization/images/actor-analytisc.png diff --git a/sources/platform/actors/publishing/images/actor-badge/deprecated.svg b/sources/platform/6-publishing-and-monetization/images/actor-badge/deprecated.svg similarity index 100% rename from sources/platform/actors/publishing/images/actor-badge/deprecated.svg rename to sources/platform/6-publishing-and-monetization/images/actor-badge/deprecated.svg diff --git a/sources/platform/actors/publishing/images/actor-badge/not-found.svg b/sources/platform/6-publishing-and-monetization/images/actor-badge/not-found.svg similarity index 100% rename from sources/platform/actors/publishing/images/actor-badge/not-found.svg rename to sources/platform/6-publishing-and-monetization/images/actor-badge/not-found.svg diff --git a/sources/platform/actors/publishing/images/actor-badge/ok.svg b/sources/platform/6-publishing-and-monetization/images/actor-badge/ok.svg similarity index 100% rename from sources/platform/actors/publishing/images/actor-badge/ok.svg rename to sources/platform/6-publishing-and-monetization/images/actor-badge/ok.svg diff --git a/sources/platform/actors/publishing/images/actor-badge/under-maintenance.svg b/sources/platform/6-publishing-and-monetization/images/actor-badge/under-maintenance.svg similarity index 100% rename from sources/platform/actors/publishing/images/actor-badge/under-maintenance.svg rename to sources/platform/6-publishing-and-monetization/images/actor-badge/under-maintenance.svg diff --git a/sources/platform/actors/publishing/images/actor-display-information.webp b/sources/platform/6-publishing-and-monetization/images/actor-display-information.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-display-information.webp rename to sources/platform/6-publishing-and-monetization/images/actor-display-information.webp diff --git a/sources/platform/actors/publishing/images/actor-insights.png b/sources/platform/6-publishing-and-monetization/images/actor-insights.png similarity index 100% rename from sources/platform/actors/publishing/images/actor-insights.png rename to sources/platform/6-publishing-and-monetization/images/actor-insights.png diff --git a/sources/platform/actors/publishing/images/actor-page.webp b/sources/platform/6-publishing-and-monetization/images/actor-page.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-page.webp rename to sources/platform/6-publishing-and-monetization/images/actor-page.webp diff --git a/sources/platform/actors/publishing/images/actor-publication-settings.webp b/sources/platform/6-publishing-and-monetization/images/actor-publication-settings.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-publication-settings.webp rename to sources/platform/6-publishing-and-monetization/images/actor-publication-settings.webp diff --git a/sources/platform/actors/publishing/images/actor-test.webp b/sources/platform/6-publishing-and-monetization/images/actor-test.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-test.webp rename to sources/platform/6-publishing-and-monetization/images/actor-test.webp diff --git a/sources/platform/actors/publishing/images/actor-title-description.webp b/sources/platform/6-publishing-and-monetization/images/actor-title-description.webp similarity index 100% rename from sources/platform/actors/publishing/images/actor-title-description.webp rename to sources/platform/6-publishing-and-monetization/images/actor-title-description.webp diff --git a/sources/platform/actors/publishing/images/actor_analytics.png b/sources/platform/6-publishing-and-monetization/images/actor_analytics.png similarity index 100% rename from sources/platform/actors/publishing/images/actor_analytics.png rename to sources/platform/6-publishing-and-monetization/images/actor_analytics.png diff --git a/sources/platform/actors/publishing/images/actor_insights.png b/sources/platform/6-publishing-and-monetization/images/actor_insights.png similarity index 100% rename from sources/platform/actors/publishing/images/actor_insights.png rename to sources/platform/6-publishing-and-monetization/images/actor_insights.png diff --git a/sources/platform/actors/publishing/images/apify-actor-start.png b/sources/platform/6-publishing-and-monetization/images/apify-actor-start.png similarity index 100% rename from sources/platform/actors/publishing/images/apify-actor-start.png rename to sources/platform/6-publishing-and-monetization/images/apify-actor-start.png diff --git a/sources/platform/actors/publishing/images/apify-store.webp b/sources/platform/6-publishing-and-monetization/images/apify-store.webp similarity index 100% rename from sources/platform/actors/publishing/images/apify-store.webp rename to sources/platform/6-publishing-and-monetization/images/apify-store.webp diff --git a/sources/platform/actors/publishing/images/github-badge-screenshot.png b/sources/platform/6-publishing-and-monetization/images/github-badge-screenshot.png similarity index 100% rename from sources/platform/actors/publishing/images/github-badge-screenshot.png rename to sources/platform/6-publishing-and-monetization/images/github-badge-screenshot.png diff --git a/sources/platform/actors/publishing/images/monetization-section.png b/sources/platform/6-publishing-and-monetization/images/monetization-section.png similarity index 100% rename from sources/platform/actors/publishing/images/monetization-section.png rename to sources/platform/6-publishing-and-monetization/images/monetization-section.png diff --git a/sources/platform/actors/publishing/images/monetization_section.png b/sources/platform/6-publishing-and-monetization/images/monetization_section.png similarity index 100% rename from sources/platform/actors/publishing/images/monetization_section.png rename to sources/platform/6-publishing-and-monetization/images/monetization_section.png diff --git a/sources/platform/actors/publishing/images/monetization_wizard.png b/sources/platform/6-publishing-and-monetization/images/monetization_wizard.png similarity index 100% rename from sources/platform/actors/publishing/images/monetization_wizard.png rename to sources/platform/6-publishing-and-monetization/images/monetization_wizard.png diff --git a/sources/platform/actors/publishing/images/monetize_actor_set_up_monetization.png b/sources/platform/6-publishing-and-monetization/images/monetize_actor_set_up_monetization.png similarity index 100% rename from sources/platform/actors/publishing/images/monetize_actor_set_up_monetization.png rename to sources/platform/6-publishing-and-monetization/images/monetize_actor_set_up_monetization.png diff --git a/sources/platform/actors/publishing/images/ppe-wizard.png b/sources/platform/6-publishing-and-monetization/images/ppe-wizard.png similarity index 100% rename from sources/platform/actors/publishing/images/ppe-wizard.png rename to sources/platform/6-publishing-and-monetization/images/ppe-wizard.png diff --git a/sources/platform/actors/publishing/images/ppr-wizard.png b/sources/platform/6-publishing-and-monetization/images/ppr-wizard.png similarity index 100% rename from sources/platform/actors/publishing/images/ppr-wizard.png rename to sources/platform/6-publishing-and-monetization/images/ppr-wizard.png diff --git a/sources/platform/actors/publishing/images/publish-actor-to-store.webp b/sources/platform/6-publishing-and-monetization/images/publish-actor-to-store.webp similarity index 100% rename from sources/platform/actors/publishing/images/publish-actor-to-store.webp rename to sources/platform/6-publishing-and-monetization/images/publish-actor-to-store.webp diff --git a/sources/platform/actors/publishing/images/rental-wizard.png b/sources/platform/6-publishing-and-monetization/images/rental-wizard.png similarity index 100% rename from sources/platform/actors/publishing/images/rental-wizard.png rename to sources/platform/6-publishing-and-monetization/images/rental-wizard.png diff --git a/sources/platform/actors/publishing/index.mdx b/sources/platform/6-publishing-and-monetization/index.mdx similarity index 90% rename from sources/platform/actors/publishing/index.mdx rename to sources/platform/6-publishing-and-monetization/index.mdx index 465f488614..d28ac8f471 100644 --- a/sources/platform/actors/publishing/index.mdx +++ b/sources/platform/6-publishing-and-monetization/index.mdx @@ -1,8 +1,8 @@ --- -title: Publishing and monetization -description: Learn about publishing, and monetizing your Actors on the Apify platform. -sidebar_position: 7.5 -slug: /actors/publishing +title: "Publishing & monetization" +description: "Share Actors and earn from the creator economy" +sidebar_position: 1 +slug: /publishing-and-monetization --- **Apify provides a platform for developing, publishing, and monetizing web automation solutions called Actors. This guide covers the key stages involved in publishing and monetizing your Actors on the Apify platform.** @@ -16,9 +16,9 @@ slug: /actors/publishing To build & publish an Actor on [Apify Store](https://apify.com/store), you'll go through the following main stages: -1. [Development](../development/index.md). +1. [Development](/platform/building-actors). 2. [Publication](./publishing/publish) and set up of [monetization](./publishing/monetize). -3. [Testing](../development/automated_tests.md). +3. [Testing](/platform/building-actors/automated-tests). 4. [Promotion](https://apify.notion.site/3fdc9fd4c8164649a2024c9ca7a2d0da?v=6d262c0b026d49bfa45771cd71f8c9ab). ## Benefits of publishing and monetization @@ -41,7 +41,7 @@ Packaging your software as an Actor allows you to launch new SaaS product faster - Pay-per-result for usage-based pricing - Pay-per-event for specific operations -To learn more visit our [Actors in Store](https://docs.apify.com/platform/actors/running/actors-in-store#pricing-models) page. +To learn more visit our [Actors in Store](https://docs.apify.com/platform/using-actors/running/actors-in-store#pricing-models) page. ## Maintain public Actors @@ -65,7 +65,7 @@ While refactoring and updating your Actor's code is encouraged, be cautious of m Pay special attention to your Actor's documentation ([README](https://apify.notion.site/How-to-create-an-Actor-README-759a1614daa54bee834ee39fe4d98bc2)). It should be clear, detailed, concise and, readable, using simple language and avoiding technical jargon whenever possible, as your users may not be developers. -Ensure periodic testing, either manually or by [setting up automatic testing](../development/automated_tests.md) and [monitoring](https://apify.com/apify/monitoring). This can help prevent users from encountering issues with your Actor. +Ensure periodic testing, either manually or by [setting up automatic testing](/platform/building-actors/automated-tests) and [monitoring](https://apify.com/apify/monitoring). This can help prevent users from encountering issues with your Actor. ## Inspiration for new Actors diff --git a/sources/platform/actors/publishing/monetize/index.mdx b/sources/platform/6-publishing-and-monetization/monetize/index.mdx similarity index 98% rename from sources/platform/actors/publishing/monetize/index.mdx rename to sources/platform/6-publishing-and-monetization/monetize/index.mdx index 5e0d8d9e1d..f5e04840a5 100644 --- a/sources/platform/actors/publishing/monetize/index.mdx +++ b/sources/platform/6-publishing-and-monetization/monetize/index.mdx @@ -23,7 +23,7 @@ Actors in Apify Store can be published under one of the following pricing models 1. _Pay per result (PPR)_: Users don't pay for the platform usage costs. Instead, they pay the developer based on the number of results produced by the Actor. 1. _Pay per event (PPE)_: Users don't pay for the platform usage cost the Actor generates. Instead, they pay based on specific events that are programmatically triggered from the Actor's source code. These events are defined by the developer and can include actions such as generating a single result or starting an Actor. -For a detailed comparison of pricing models from the perspective of your users, refer to [Actors in Store](/platform/actors/running/actors-in-store) page. +For a detailed comparison of pricing models from the perspective of your users, refer to [Actors in Store](/platform/using-actors/running/actors-in-store) page. ## Key benefits diff --git a/sources/platform/actors/publishing/monetize/pay_per_event.mdx b/sources/platform/6-publishing-and-monetization/monetize/pay_per_event.mdx similarity index 96% rename from sources/platform/actors/publishing/monetize/pay_per_event.mdx rename to sources/platform/6-publishing-and-monetization/monetize/pay_per_event.mdx index fabacead53..7bfcd73080 100644 --- a/sources/platform/actors/publishing/monetize/pay_per_event.mdx +++ b/sources/platform/6-publishing-and-monetization/monetize/pay_per_event.mdx @@ -1,7 +1,7 @@ --- title: Pay per event description: Learn how to monetize your Actor with pay-per-event (PPE) pricing, charging users for specific actions like Actor starts, dataset items, or API calls, and understand how to set profitable, transparent event-based pricing. -slug: /actors/publishing/monetize/pay-per-event +slug: /publishing-and-monetization/monetize/pay-per-event sidebar_position: 3 --- @@ -65,7 +65,7 @@ The `eventChargeLimitReached` property checks if the current event type can be c :::info ACTOR_MAX_TOTAL_CHARGE_USD environment variable -For pay-per-event Actors, users set a spending limit through the Apify Console. This limit is available in your Actor code as the `ACTOR_MAX_TOTAL_CHARGE_USD` [environment variable](/platform/actors/development/programming-interface/environment-variables), which contains the user's maximum cost. +For pay-per-event Actors, users set a spending limit through the Apify Console. This limit is available in your Actor code as the `ACTOR_MAX_TOTAL_CHARGE_USD` [environment variable](/platform/using-actors/development/programming-interface/environment-variables), which contains the user's maximum cost. ::: @@ -138,7 +138,7 @@ You can also choose not to use it, but then you must handle API integration and ### Set memory limits -Set memory limits using `minMemoryMbytes` and `maxMemoryMbytes` in your [`actor.json`](https://docs.apify.com/platform/actors/development/actor-definition/actor-json) file to control platform usage costs. +Set memory limits using `minMemoryMbytes` and `maxMemoryMbytes` in your [`actor.json`](https://docs.apify.com/platform/using-actors/development/actor-definition/actor-json) file to control platform usage costs. ```json { @@ -326,4 +326,4 @@ To implement PPE pricing, you need to define specific events in your Actor code. ## Next steps -- Check out the [Pricing and costs](./pricing_and_costs.mdx) section to learn how to compute your costs. +- Check out the [Pricing and costs](/platform/publishing-and-monetization/monetize/pricing-and-costs) section to learn how to compute your costs. diff --git a/sources/platform/actors/publishing/monetize/pay_per_result.mdx b/sources/platform/6-publishing-and-monetization/monetize/pay_per_result.mdx similarity index 96% rename from sources/platform/actors/publishing/monetize/pay_per_result.mdx rename to sources/platform/6-publishing-and-monetization/monetize/pay_per_result.mdx index 09ceea1331..7dc530ba24 100644 --- a/sources/platform/actors/publishing/monetize/pay_per_result.mdx +++ b/sources/platform/6-publishing-and-monetization/monetize/pay_per_result.mdx @@ -1,7 +1,7 @@ --- title: Pay per result description: Learn how to monetize your Actor with pay-per-result (PPR) pricing, charging users based on the number of results produced and stored in the dataset, and understand how to set profitable, transparent result-based pricing. -slug: /actors/publishing/monetize/pay-per-result +slug: /publishing-and-monetization/monetize/pay-per-result sidebar_position: 2 --- @@ -41,7 +41,7 @@ To ensure profitability, check the following best practices. ### Set memory limits -Set memory limits using `minMemoryMbytes` and `maxMemoryMbytes` in your [`actor.json`](https://docs.apify.com/platform/actors/development/actor-definition/actor-json) file to control platform usage costs. +Set memory limits using `minMemoryMbytes` and `maxMemoryMbytes` in your [`actor.json`](https://docs.apify.com/platform/using-actors/development/actor-definition/actor-json) file to control platform usage costs. ```json { @@ -217,4 +217,4 @@ Your profit is computed only from the first two users, since they are on Apify p ## Next steps -- Check out the [Pricing and costs](./pricing_and_costs.mdx) section to learn how to compute your costs. +- Check out the [Pricing and costs](/platform/publishing-and-monetization/monetize/pricing-and-costs) section to learn how to compute your costs. diff --git a/sources/platform/actors/publishing/monetize/pricing_and_costs.mdx b/sources/platform/6-publishing-and-monetization/monetize/pricing_and_costs.mdx similarity index 98% rename from sources/platform/actors/publishing/monetize/pricing_and_costs.mdx rename to sources/platform/6-publishing-and-monetization/monetize/pricing_and_costs.mdx index 4ec96447ac..14f3c97f30 100644 --- a/sources/platform/actors/publishing/monetize/pricing_and_costs.mdx +++ b/sources/platform/6-publishing-and-monetization/monetize/pricing_and_costs.mdx @@ -1,7 +1,7 @@ --- title: Pricing and costs description: Learn how to set Actor pricing and calculate your costs, including platform usage rates, discount tiers, and profit formulas for PPE and PPR monetization models. -slug: /actors/publishing/monetize/pricing-and-costs +slug: /publishing-and-monetization/monetize/pricing-and-costs sidebar_position: 3 --- diff --git a/sources/platform/actors/publishing/monetize/rental.mdx b/sources/platform/6-publishing-and-monetization/monetize/rental.mdx similarity index 91% rename from sources/platform/actors/publishing/monetize/rental.mdx rename to sources/platform/6-publishing-and-monetization/monetize/rental.mdx index 204cd1d74d..5de2d3d0e1 100644 --- a/sources/platform/actors/publishing/monetize/rental.mdx +++ b/sources/platform/6-publishing-and-monetization/monetize/rental.mdx @@ -1,7 +1,7 @@ --- title: Rental pricing model description: Learn how to monetize your Actor with the rental pricing model, offering users a free trial and a flat monthly fee, and understand how profit is calculated and the limitations of this approach. -slug: /actors/publishing/monetize/rental +slug: /publishing-and-monetization/monetize/rental sidebar_position: 1 --- @@ -39,7 +39,7 @@ The growing limitation is AI compatibility. [Apify's MCP server](/platform/integ ## Consider pay-per-result or pay-per-event pricing models -We recommend using the [pay-per-result](/platform/actors/publishing/monetize/pay-per-result) or [pay-per-event](/platform/actors/publishing/monetize/pay-per-event) models instead. +We recommend using the [pay-per-result](/platform/publishing-and-monetization/publish/monetize/pay-per-result) or [pay-per-event](/platform/publishing-and-monetization/publish/monetize/pay-per-event) models instead. ## Example of a rental pricing model diff --git a/sources/platform/actors/publishing/publish.mdx b/sources/platform/6-publishing-and-monetization/publish.mdx similarity index 98% rename from sources/platform/actors/publishing/publish.mdx rename to sources/platform/6-publishing-and-monetization/publish.mdx index 109a9dc219..69fb90c762 100644 --- a/sources/platform/actors/publishing/publish.mdx +++ b/sources/platform/6-publishing-and-monetization/publish.mdx @@ -1,7 +1,7 @@ --- title: Publish your Actor description: Prepare your Actor for Apify Store with a description and README file, and learn how to make your Actor available to the public. -slug: /actors/publishing/publish +slug: /publishing-and-monetization/publish sidebar_position: 1 --- diff --git a/sources/platform/actors/publishing/testing.mdx b/sources/platform/6-publishing-and-monetization/testing.mdx similarity index 88% rename from sources/platform/actors/publishing/testing.mdx rename to sources/platform/6-publishing-and-monetization/testing.mdx index fb9580d44d..afbbca1705 100644 --- a/sources/platform/actors/publishing/testing.mdx +++ b/sources/platform/6-publishing-and-monetization/testing.mdx @@ -1,7 +1,7 @@ --- title: Automated testing description: Apify has a QA system that regularly runs automated tests to ensure that all Actors in the store are functional. -slug: /actors/publishing/test +slug: /publishing-and-monetization/test sidebar_position: 3 --- @@ -16,7 +16,7 @@ This helps us to flag Actors that temporarily don't work as expected `under main ### How we test -The test runs the Actor with its default input (defined by the [**prefill**](https://docs.apify.com/platform/actors/development/actor-definition/input-schema/specification/v1#prefill-vs-default-vs-required) option in the input schema file) +The test runs the Actor with its default input (defined by the [**prefill**](https://docs.apify.com/platform/using-actors/development/actor-definition/input-schema/specification/v1#prefill-vs-default-vs-required) option in the input schema file) and expects it to finish with a **Succeeded** status and non-empty default dataset within 5 minutes of the beginning of the run. ![Actor page](./images/actor-test.webp) @@ -39,4 +39,4 @@ If that's the case with your Actor, please contact support at [support@apify.com You can easily implement your own tests and customize them to fit your Actor's particularities by using our public [Actor Testing](https://apify.com/pocesar/actor-testing) tool available in Apify Store. -For more information, see the [automated testing](../development/automated_tests.md) section. +For more information, see the [automated testing](/platform/building-actors/automated-tests) section. diff --git a/sources/platform/7-integrations/_category_.yml b/sources/platform/7-integrations/_category_.yml new file mode 100644 index 0000000000..23102780c5 --- /dev/null +++ b/sources/platform/7-integrations/_category_.yml @@ -0,0 +1,2 @@ +label: 'Integrations' +position: 7 \ No newline at end of file diff --git a/sources/platform/integrations/actors/_category_.yml b/sources/platform/7-integrations/actors/_category_.yml similarity index 100% rename from sources/platform/integrations/actors/_category_.yml rename to sources/platform/7-integrations/actors/_category_.yml diff --git a/sources/platform/integrations/actors/images/integration_triggers.png b/sources/platform/7-integrations/actors/images/integration_triggers.png similarity index 100% rename from sources/platform/integrations/actors/images/integration_triggers.png rename to sources/platform/7-integrations/actors/images/integration_triggers.png diff --git a/sources/platform/integrations/actors/images/integrations_add.png b/sources/platform/7-integrations/actors/images/integrations_add.png similarity index 100% rename from sources/platform/integrations/actors/images/integrations_add.png rename to sources/platform/7-integrations/actors/images/integrations_add.png diff --git a/sources/platform/integrations/actors/images/integrations_test_options.png b/sources/platform/7-integrations/actors/images/integrations_test_options.png similarity index 100% rename from sources/platform/integrations/actors/images/integrations_test_options.png rename to sources/platform/7-integrations/actors/images/integrations_test_options.png diff --git a/sources/platform/integrations/actors/images/specific_vs_generic_integrations.png b/sources/platform/7-integrations/actors/images/specific_vs_generic_integrations.png similarity index 100% rename from sources/platform/integrations/actors/images/specific_vs_generic_integrations.png rename to sources/platform/7-integrations/actors/images/specific_vs_generic_integrations.png diff --git a/sources/platform/integrations/actors/index.md b/sources/platform/7-integrations/actors/index.md similarity index 100% rename from sources/platform/integrations/actors/index.md rename to sources/platform/7-integrations/actors/index.md diff --git a/sources/platform/integrations/actors/integrating_actors_via_api.md b/sources/platform/7-integrations/actors/integrating_actors_via_api.md similarity index 100% rename from sources/platform/integrations/actors/integrating_actors_via_api.md rename to sources/platform/7-integrations/actors/integrating_actors_via_api.md diff --git a/sources/platform/integrations/actors/integration_ready_actors.md b/sources/platform/7-integrations/actors/integration_ready_actors.md similarity index 94% rename from sources/platform/integrations/actors/integration_ready_actors.md rename to sources/platform/7-integrations/actors/integration_ready_actors.md index b4f0e66fa2..035c9421d9 100644 --- a/sources/platform/integrations/actors/integration_ready_actors.md +++ b/sources/platform/7-integrations/actors/integration_ready_actors.md @@ -90,7 +90,7 @@ In the above example, we're focusing on accessing a run's default dataset, but t ## Making your Actor available to other users -To allow other users to use your Actor as an integration, all you need to do is [publish it in Apify Store](/platform/actors/publishing), so users can then integrate it using the **Connect Actor or task** button on the **Integrations** tab of any Actor. While publishing the Actor is enough, there are two ways to make it more visible to users. +To allow other users to use your Actor as an integration, all you need to do is [publish it in Apify Store](/platform/publishing-and-monetization), so users can then integrate it using the **Connect Actor or task** button on the **Integrations** tab of any Actor. While publishing the Actor is enough, there are two ways to make it more visible to users. For Actors that are generic enough to be used with most other Actors, it's possible to have them listed under **Generic integrations** in the **Integrations** tab. This includes (but is not limited to) Actors that upload datasets to databases, send notifications through various messaging systems, create issues in ticketing systems, etc. To have your Actor listed under our generic integrations, [contact support](mailto:support@apify.com?subject=Actor%20generic%20integration). diff --git a/sources/platform/integrations/ai/_category_.yml b/sources/platform/7-integrations/ai/_category_.yml similarity index 100% rename from sources/platform/integrations/ai/_category_.yml rename to sources/platform/7-integrations/ai/_category_.yml diff --git a/sources/platform/integrations/ai/agno.md b/sources/platform/7-integrations/ai/agno.md similarity index 100% rename from sources/platform/integrations/ai/agno.md rename to sources/platform/7-integrations/ai/agno.md diff --git a/sources/platform/integrations/ai/aws_bedrock.md b/sources/platform/7-integrations/ai/aws_bedrock.md similarity index 98% rename from sources/platform/integrations/ai/aws_bedrock.md rename to sources/platform/7-integrations/ai/aws_bedrock.md index 35b94944c6..18992c9f57 100644 --- a/sources/platform/integrations/ai/aws_bedrock.md +++ b/sources/platform/7-integrations/ai/aws_bedrock.md @@ -25,7 +25,7 @@ Amazon Bedrock allows you to create AI agents powered by large language models t Before getting started, ensure you have: - An active AWS Account. -- An Apify account and an [API token](https://docs.apify.com/platform/integrations/api#api-token). +- An Apify account and an [API token](https://docs.apify.com///platform/integrations/api#api-token). - Granted access to any Large Language Model from Amazon Bedrock. To add access to a LLM, follow this [guide](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-modify.html). We'll use **Anthropic Claude 3.5 Sonnet** in this example. The overall process for creating an agent includes the following [steps](https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html): diff --git a/sources/platform/integrations/ai/crewai.md b/sources/platform/7-integrations/ai/crewai.md similarity index 99% rename from sources/platform/integrations/ai/crewai.md rename to sources/platform/7-integrations/ai/crewai.md index bee4e93eaa..e45c47985e 100644 --- a/sources/platform/integrations/ai/crewai.md +++ b/sources/platform/7-integrations/ai/crewai.md @@ -203,7 +203,7 @@ print(result) ## Resources -- [Apify Actors](https://docs.apify.com/platform/actors) +- [Apify Actors](https://docs.apify.com/platform/using-actors) - [CrewAI Documentation](https://docs.crewai.com/) - [What are AI agents?](https://blog.apify.com/what-are-ai-agents/) - [How to build an AI agent](https://blog.apify.com/how-to-build-an-ai-agent/) diff --git a/sources/platform/integrations/ai/flowise.md b/sources/platform/7-integrations/ai/flowise.md similarity index 100% rename from sources/platform/integrations/ai/flowise.md rename to sources/platform/7-integrations/ai/flowise.md diff --git a/sources/platform/integrations/ai/haystack.md b/sources/platform/7-integrations/ai/haystack.md similarity index 100% rename from sources/platform/integrations/ai/haystack.md rename to sources/platform/7-integrations/ai/haystack.md diff --git a/sources/platform/integrations/ai/images/lindy/lindy-action.png b/sources/platform/7-integrations/ai/images/lindy/lindy-action.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-action.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-action.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-instagram-actor.png b/sources/platform/7-integrations/ai/images/lindy/lindy-instagram-actor.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-instagram-actor.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-instagram-actor.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-new-button.png b/sources/platform/7-integrations/ai/images/lindy/lindy-new-button.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-new-button.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-new-button.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-received.png b/sources/platform/7-integrations/ai/images/lindy/lindy-received.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-received.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-received.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-run-actor.png b/sources/platform/7-integrations/ai/images/lindy/lindy-run-actor.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-run-actor.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-run-actor.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-scratch.png b/sources/platform/7-integrations/ai/images/lindy/lindy-scratch.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-scratch.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-scratch.png diff --git a/sources/platform/integrations/ai/images/lindy/lindy-trigger.png b/sources/platform/7-integrations/ai/images/lindy/lindy-trigger.png similarity index 100% rename from sources/platform/integrations/ai/images/lindy/lindy-trigger.png rename to sources/platform/7-integrations/ai/images/lindy/lindy-trigger.png diff --git a/sources/platform/integrations/ai/langchain.md b/sources/platform/7-integrations/ai/langchain.md similarity index 100% rename from sources/platform/integrations/ai/langchain.md rename to sources/platform/7-integrations/ai/langchain.md diff --git a/sources/platform/integrations/ai/langflow.md b/sources/platform/7-integrations/ai/langflow.md similarity index 100% rename from sources/platform/integrations/ai/langflow.md rename to sources/platform/7-integrations/ai/langflow.md diff --git a/sources/platform/integrations/ai/langgraph.md b/sources/platform/7-integrations/ai/langgraph.md similarity index 98% rename from sources/platform/integrations/ai/langgraph.md rename to sources/platform/7-integrations/ai/langgraph.md index 8690ed08f8..c0e3ced2a7 100644 --- a/sources/platform/integrations/ai/langgraph.md +++ b/sources/platform/7-integrations/ai/langgraph.md @@ -156,5 +156,5 @@ for state in agent_executor.stream( ## Resources -- [Apify Actors](https://docs.apify.com/platform/actors) +- [Apify Actors](https://docs.apify.com/platform/using-actors) - [LangGraph - How to Create a ReAct Agent](https://langchain-ai.github.io/langgraph/how-tos/create-react-agent/) diff --git a/sources/platform/integrations/ai/lindy.md b/sources/platform/7-integrations/ai/lindy.md similarity index 97% rename from sources/platform/integrations/ai/lindy.md rename to sources/platform/7-integrations/ai/lindy.md index ccf796b9f8..919dde62e6 100644 --- a/sources/platform/integrations/ai/lindy.md +++ b/sources/platform/7-integrations/ai/lindy.md @@ -51,7 +51,7 @@ This section demonstrates how to integrate Apify's data extraction capabilities :::tip Actor Availability -You have access to thousands of Actors available on the [Apify Store](https://apify.com/store). Please note that Actors using the _rental pricing model_ are not available for use with this integration. For details on Actor pricing models, refer to our [Pricing Documentation](/platform/actors/publishing/monetize#rental-pricing-model). +You have access to thousands of Actors available on the [Apify Store](https://apify.com/store). Please note that Actors using the _rental pricing model_ are not available for use with this integration. For details on Actor pricing models, refer to our [Pricing Documentation](/platform/publishing-and-monetization/publish/monetize#rental-pricing-model). ::: diff --git a/sources/platform/integrations/ai/llama.md b/sources/platform/7-integrations/ai/llama.md similarity index 96% rename from sources/platform/integrations/ai/llama.md rename to sources/platform/7-integrations/ai/llama.md index 3a7a642a04..b1eea590f5 100644 --- a/sources/platform/integrations/ai/llama.md +++ b/sources/platform/7-integrations/ai/llama.md @@ -28,7 +28,7 @@ After successfully installing all dependencies, we can start writing Python code ### Apify Actor -To use the Apify Actor, import `ApifyActor` and `Document`, and set your [Apify API token](https://docs.apify.com/platform/integrations/api#api-token) in the code. +To use the Apify Actor, import `ApifyActor` and `Document`, and set your [Apify API token](https://docs.apify.com///platform/integrations/api#api-token) in the code. The following example uses the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor to crawl an entire website, which will extract text content from the web pages. The extracted text is formatted as a llama_index `Document` and can be fed to a vector store or language model like GPT. diff --git a/sources/platform/integrations/ai/mastra.md b/sources/platform/7-integrations/ai/mastra.md similarity index 97% rename from sources/platform/integrations/ai/mastra.md rename to sources/platform/7-integrations/ai/mastra.md index e042feac21..cb7c14413e 100644 --- a/sources/platform/integrations/ai/mastra.md +++ b/sources/platform/7-integrations/ai/mastra.md @@ -22,7 +22,7 @@ Check out the [Mastra docs](https://mastra.ai/docs) for more information. ## What is MCP server -A [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server exposes specific data sources or tools to agents via a standardized protocol. It acts as a bridge, connecting large language models (LLMs) to external systems like databases, APIs, or local filesystems. Built on a client-server architecture, MCP servers enable secure, real-time interaction, allowing agents to fetch context or execute actions without custom integrations. Think of it as a modular plugin system for agents, simplifying how they access and process data. Apify provides [Actors MCP Server](https://apify.com/apify/actors-mcp-server) to expose [Apify Actors](https://docs.apify.com/platform/actors) from the [Apify Store](https://apify.com/store) as tools via the MCP protocol. +A [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server exposes specific data sources or tools to agents via a standardized protocol. It acts as a bridge, connecting large language models (LLMs) to external systems like databases, APIs, or local filesystems. Built on a client-server architecture, MCP servers enable secure, real-time interaction, allowing agents to fetch context or execute actions without custom integrations. Think of it as a modular plugin system for agents, simplifying how they access and process data. Apify provides [Actors MCP Server](https://apify.com/apify/actors-mcp-server) to expose [Apify Actors](https://docs.apify.com/platform/using-actors) from the [Apify Store](https://apify.com/store) as tools via the MCP protocol. ## How to use Apify with Mastra via MCP @@ -214,7 +214,7 @@ await mcpClient.disconnect(); ## Resources -- [Apify Actors](https://docs.apify.com/platform/actors) +- [Apify Actors](https://docs.apify.com/platform/using-actors) - [Mastra Documentation](https://mastra.ai/docs) - [Apify MCP Server](https://mcp.apify.com) - [How to use MCP with Apify Actors](https://blog.apify.com/how-to-use-mcp/) diff --git a/sources/platform/integrations/ai/mcp.md b/sources/platform/7-integrations/ai/mcp.md similarity index 100% rename from sources/platform/integrations/ai/mcp.md rename to sources/platform/7-integrations/ai/mcp.md diff --git a/sources/platform/integrations/ai/milvus.md b/sources/platform/7-integrations/ai/milvus.md similarity index 97% rename from sources/platform/integrations/ai/milvus.md rename to sources/platform/7-integrations/ai/milvus.md index 0ed1e293b5..e75bc4ba51 100644 --- a/sources/platform/integrations/ai/milvus.md +++ b/sources/platform/7-integrations/ai/milvus.md @@ -24,7 +24,7 @@ Before you begin, ensure that you have the following: - A Milvus/Zilliz database universal resource identifier (URI) and Token to setup the client. Optionally, you can use a username and password in the URI. You can run Milvus on Docker or Kubernetes, but in this example, we'll use the hosted Milvus service at [Zilliz Cloud](https://cloud.zilliz.com). - An [OpenAI API key](https://openai.com/index/openai-api/) to compute text embeddings. -- An [Apify API token](https://docs.apify.com/platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). +- An [Apify API token](https://docs.apify.com///platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). ### How to set up Milvus/Zilliz database diff --git a/sources/platform/integrations/ai/openai_assistants.md b/sources/platform/7-integrations/ai/openai_assistants.md similarity index 100% rename from sources/platform/integrations/ai/openai_assistants.md rename to sources/platform/7-integrations/ai/openai_assistants.md diff --git a/sources/platform/integrations/ai/pinecone.md b/sources/platform/7-integrations/ai/pinecone.md similarity index 97% rename from sources/platform/integrations/ai/pinecone.md rename to sources/platform/7-integrations/ai/pinecone.md index e87b447cda..4dd41a542e 100644 --- a/sources/platform/integrations/ai/pinecone.md +++ b/sources/platform/7-integrations/ai/pinecone.md @@ -23,7 +23,7 @@ Before you begin, ensure that you have the following: - A [Pinecone database](https://www.pinecone.io/) and index set up. - A Pinecone index created & Pinecone API token obtained. - An [OpenAI API key](https://openai.com/index/openai-api/) to compute text embeddings. -- An [Apify API token](https://docs.apify.com/platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). +- An [Apify API token](https://docs.apify.com///platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). ### How to setup Pinecone database and create an index diff --git a/sources/platform/integrations/ai/qdrant.md b/sources/platform/7-integrations/ai/qdrant.md similarity index 97% rename from sources/platform/integrations/ai/qdrant.md rename to sources/platform/7-integrations/ai/qdrant.md index 66f4e74c0e..ab37b7c49f 100644 --- a/sources/platform/integrations/ai/qdrant.md +++ b/sources/platform/7-integrations/ai/qdrant.md @@ -23,7 +23,7 @@ Before you begin, ensure that you have the following: - A [Qdrant cluster](https://qdrant.tech) set up. - A Qdrant URL to the database and Qdrant API token. - An [OpenAI API key](https://openai.com/index/openai-api/) to compute text embeddings. -- An [Apify API token](https://docs.apify.com/platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). +- An [Apify API token](https://docs.apify.com///platform/integrations/api#api-token) to access [Apify Actors](https://apify.com/store). ### How to setup Qdrant database and create a cluster diff --git a/sources/platform/integrations/data-storage/_category_.yml b/sources/platform/7-integrations/data-storage/_category_.yml similarity index 100% rename from sources/platform/integrations/data-storage/_category_.yml rename to sources/platform/7-integrations/data-storage/_category_.yml diff --git a/sources/platform/integrations/data-storage/airbyte.md b/sources/platform/7-integrations/data-storage/airbyte.md similarity index 100% rename from sources/platform/integrations/data-storage/airbyte.md rename to sources/platform/7-integrations/data-storage/airbyte.md diff --git a/sources/platform/integrations/data-storage/airtable.md b/sources/platform/7-integrations/data-storage/airtable.md similarity index 100% rename from sources/platform/integrations/data-storage/airtable.md rename to sources/platform/7-integrations/data-storage/airtable.md diff --git a/sources/platform/integrations/data-storage/drive.md b/sources/platform/7-integrations/data-storage/drive.md similarity index 100% rename from sources/platform/integrations/data-storage/drive.md rename to sources/platform/7-integrations/data-storage/drive.md diff --git a/sources/platform/integrations/data-storage/keboola.md b/sources/platform/7-integrations/data-storage/keboola.md similarity index 100% rename from sources/platform/integrations/data-storage/keboola.md rename to sources/platform/7-integrations/data-storage/keboola.md diff --git a/sources/platform/integrations/images/airbyte-source-setup.png b/sources/platform/7-integrations/images/airbyte-source-setup.png similarity index 100% rename from sources/platform/integrations/images/airbyte-source-setup.png rename to sources/platform/7-integrations/images/airbyte-source-setup.png diff --git a/sources/platform/integrations/images/airbyte-sources-web.png b/sources/platform/7-integrations/images/airbyte-sources-web.png similarity index 100% rename from sources/platform/integrations/images/airbyte-sources-web.png rename to sources/platform/7-integrations/images/airbyte-sources-web.png diff --git a/sources/platform/integrations/images/airbyte-sources.png b/sources/platform/7-integrations/images/airbyte-sources.png similarity index 100% rename from sources/platform/integrations/images/airbyte-sources.png rename to sources/platform/7-integrations/images/airbyte-sources.png diff --git a/sources/platform/integrations/images/airtable/connect-account-1.png b/sources/platform/7-integrations/images/airtable/connect-account-1.png similarity index 100% rename from sources/platform/integrations/images/airtable/connect-account-1.png rename to sources/platform/7-integrations/images/airtable/connect-account-1.png diff --git a/sources/platform/integrations/images/airtable/connect-account-2.png b/sources/platform/7-integrations/images/airtable/connect-account-2.png similarity index 100% rename from sources/platform/integrations/images/airtable/connect-account-2.png rename to sources/platform/7-integrations/images/airtable/connect-account-2.png diff --git a/sources/platform/integrations/images/airtable/set-up-integration-1.png b/sources/platform/7-integrations/images/airtable/set-up-integration-1.png similarity index 100% rename from sources/platform/integrations/images/airtable/set-up-integration-1.png rename to sources/platform/7-integrations/images/airtable/set-up-integration-1.png diff --git a/sources/platform/integrations/images/airtable/set-up-integration-2.png b/sources/platform/7-integrations/images/airtable/set-up-integration-2.png similarity index 100% rename from sources/platform/integrations/images/airtable/set-up-integration-2.png rename to sources/platform/7-integrations/images/airtable/set-up-integration-2.png diff --git a/sources/platform/integrations/images/airtable/set-up-integration-3.png b/sources/platform/7-integrations/images/airtable/set-up-integration-3.png similarity index 100% rename from sources/platform/integrations/images/airtable/set-up-integration-3.png rename to sources/platform/7-integrations/images/airtable/set-up-integration-3.png diff --git a/sources/platform/integrations/images/api-token-expiration-date.png b/sources/platform/7-integrations/images/api-token-expiration-date.png similarity index 100% rename from sources/platform/integrations/images/api-token-expiration-date.png rename to sources/platform/7-integrations/images/api-token-expiration-date.png diff --git a/sources/platform/integrations/images/api-token-leaked.png b/sources/platform/7-integrations/images/api-token-leaked.png similarity index 100% rename from sources/platform/integrations/images/api-token-leaked.png rename to sources/platform/7-integrations/images/api-token-leaked.png diff --git a/sources/platform/integrations/images/api-token-organization.png b/sources/platform/7-integrations/images/api-token-organization.png similarity index 100% rename from sources/platform/integrations/images/api-token-organization.png rename to sources/platform/7-integrations/images/api-token-organization.png diff --git a/sources/platform/integrations/images/api-token-rotate.png b/sources/platform/7-integrations/images/api-token-rotate.png similarity index 100% rename from sources/platform/integrations/images/api-token-rotate.png rename to sources/platform/7-integrations/images/api-token-rotate.png diff --git a/sources/platform/integrations/images/api-token-scoped-default-storage-access.png b/sources/platform/7-integrations/images/api-token-scoped-default-storage-access.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-default-storage-access.png rename to sources/platform/7-integrations/images/api-token-scoped-default-storage-access.png diff --git a/sources/platform/integrations/images/api-token-scoped-dependencies.png b/sources/platform/7-integrations/images/api-token-scoped-dependencies.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-dependencies.png rename to sources/platform/7-integrations/images/api-token-scoped-dependencies.png diff --git a/sources/platform/integrations/images/api-token-scoped-restricted-access-active.png b/sources/platform/7-integrations/images/api-token-scoped-restricted-access-active.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-restricted-access-active.png rename to sources/platform/7-integrations/images/api-token-scoped-restricted-access-active.png diff --git a/sources/platform/integrations/images/api-token-scoped-run-modes.png b/sources/platform/7-integrations/images/api-token-scoped-run-modes.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-run-modes.png rename to sources/platform/7-integrations/images/api-token-scoped-run-modes.png diff --git a/sources/platform/integrations/images/api-token-scoped-run-tasks.png b/sources/platform/7-integrations/images/api-token-scoped-run-tasks.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-run-tasks.png rename to sources/platform/7-integrations/images/api-token-scoped-run-tasks.png diff --git a/sources/platform/integrations/images/api-token-scoped-with-combining-permissions.png b/sources/platform/7-integrations/images/api-token-scoped-with-combining-permissions.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped-with-combining-permissions.png rename to sources/platform/7-integrations/images/api-token-scoped-with-combining-permissions.png diff --git a/sources/platform/integrations/images/api-token-scoped.png b/sources/platform/7-integrations/images/api-token-scoped.png similarity index 100% rename from sources/platform/integrations/images/api-token-scoped.png rename to sources/platform/7-integrations/images/api-token-scoped.png diff --git a/sources/platform/integrations/images/api-token.png b/sources/platform/7-integrations/images/api-token.png similarity index 100% rename from sources/platform/integrations/images/api-token.png rename to sources/platform/7-integrations/images/api-token.png diff --git a/sources/platform/integrations/images/apify-git-provider-select.png b/sources/platform/7-integrations/images/apify-git-provider-select.png similarity index 100% rename from sources/platform/integrations/images/apify-git-provider-select.png rename to sources/platform/7-integrations/images/apify-git-provider-select.png diff --git a/sources/platform/integrations/images/apify-git-repository-account.png b/sources/platform/7-integrations/images/apify-git-repository-account.png similarity index 100% rename from sources/platform/integrations/images/apify-git-repository-account.png rename to sources/platform/7-integrations/images/apify-git-repository-account.png diff --git a/sources/platform/integrations/images/apify-git-repository-add.png b/sources/platform/7-integrations/images/apify-git-repository-add.png similarity index 100% rename from sources/platform/integrations/images/apify-git-repository-add.png rename to sources/platform/7-integrations/images/apify-git-repository-add.png diff --git a/sources/platform/integrations/images/apify-git-repository-search.png b/sources/platform/7-integrations/images/apify-git-repository-search.png similarity index 100% rename from sources/platform/integrations/images/apify-git-repository-search.png rename to sources/platform/7-integrations/images/apify-git-repository-search.png diff --git a/sources/platform/integrations/images/apify-git-repository.png b/sources/platform/7-integrations/images/apify-git-repository.png similarity index 100% rename from sources/platform/integrations/images/apify-git-repository.png rename to sources/platform/7-integrations/images/apify-git-repository.png diff --git a/sources/platform/integrations/images/apify-integrations-token.png b/sources/platform/7-integrations/images/apify-integrations-token.png similarity index 100% rename from sources/platform/integrations/images/apify-integrations-token.png rename to sources/platform/7-integrations/images/apify-integrations-token.png diff --git a/sources/platform/integrations/images/apify-make-dataset.png b/sources/platform/7-integrations/images/apify-make-dataset.png similarity index 100% rename from sources/platform/integrations/images/apify-make-dataset.png rename to sources/platform/7-integrations/images/apify-make-dataset.png diff --git a/sources/platform/integrations/images/apify-make-run-actor.png b/sources/platform/7-integrations/images/apify-make-run-actor.png similarity index 100% rename from sources/platform/integrations/images/apify-make-run-actor.png rename to sources/platform/7-integrations/images/apify-make-run-actor.png diff --git a/sources/platform/integrations/images/apify-make-run-task.png b/sources/platform/7-integrations/images/apify-make-run-task.png similarity index 100% rename from sources/platform/integrations/images/apify-make-run-task.png rename to sources/platform/7-integrations/images/apify-make-run-task.png diff --git a/sources/platform/integrations/images/apify-make-trigger.png b/sources/platform/7-integrations/images/apify-make-trigger.png similarity index 100% rename from sources/platform/integrations/images/apify-make-trigger.png rename to sources/platform/7-integrations/images/apify-make-trigger.png diff --git a/sources/platform/integrations/images/apify-module.png b/sources/platform/7-integrations/images/apify-module.png similarity index 100% rename from sources/platform/integrations/images/apify-module.png rename to sources/platform/7-integrations/images/apify-module.png diff --git a/sources/platform/integrations/images/apify-store.webp b/sources/platform/7-integrations/images/apify-store.webp similarity index 100% rename from sources/platform/integrations/images/apify-store.webp rename to sources/platform/7-integrations/images/apify-store.webp diff --git a/sources/platform/integrations/images/apify-token.png b/sources/platform/7-integrations/images/apify-token.png similarity index 100% rename from sources/platform/integrations/images/apify-token.png rename to sources/platform/7-integrations/images/apify-token.png diff --git a/sources/platform/integrations/images/aws-bedrock-agent-builder-create-action.png b/sources/platform/7-integrations/images/aws-bedrock-agent-builder-create-action.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-agent-builder-create-action.png rename to sources/platform/7-integrations/images/aws-bedrock-agent-builder-create-action.png diff --git a/sources/platform/integrations/images/aws-bedrock-agent-builder.png b/sources/platform/7-integrations/images/aws-bedrock-agent-builder.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-agent-builder.png rename to sources/platform/7-integrations/images/aws-bedrock-agent-builder.png diff --git a/sources/platform/integrations/images/aws-bedrock-agent-chat.png b/sources/platform/7-integrations/images/aws-bedrock-agent-chat.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-agent-chat.png rename to sources/platform/7-integrations/images/aws-bedrock-agent-chat.png diff --git a/sources/platform/integrations/images/aws-bedrock-ai-agent.png b/sources/platform/7-integrations/images/aws-bedrock-ai-agent.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-ai-agent.png rename to sources/platform/7-integrations/images/aws-bedrock-ai-agent.png diff --git a/sources/platform/integrations/images/aws-bedrock-create-agent.png b/sources/platform/7-integrations/images/aws-bedrock-create-agent.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-create-agent.png rename to sources/platform/7-integrations/images/aws-bedrock-create-agent.png diff --git a/sources/platform/integrations/images/aws-bedrock-menu.png b/sources/platform/7-integrations/images/aws-bedrock-menu.png similarity index 100% rename from sources/platform/integrations/images/aws-bedrock-menu.png rename to sources/platform/7-integrations/images/aws-bedrock-menu.png diff --git a/sources/platform/integrations/images/botfather-HTTP-API-blurred.png b/sources/platform/7-integrations/images/botfather-HTTP-API-blurred.png similarity index 100% rename from sources/platform/integrations/images/botfather-HTTP-API-blurred.png rename to sources/platform/7-integrations/images/botfather-HTTP-API-blurred.png diff --git a/sources/platform/integrations/images/botfather-start.png b/sources/platform/7-integrations/images/botfather-start.png similarity index 100% rename from sources/platform/integrations/images/botfather-start.png rename to sources/platform/7-integrations/images/botfather-start.png diff --git a/sources/platform/integrations/images/bubble/action_preview.png b/sources/platform/7-integrations/images/bubble/action_preview.png similarity index 100% rename from sources/platform/integrations/images/bubble/action_preview.png rename to sources/platform/7-integrations/images/bubble/action_preview.png diff --git a/sources/platform/integrations/images/bubble/add_action_to_workflow.png b/sources/platform/7-integrations/images/bubble/add_action_to_workflow.png similarity index 100% rename from sources/platform/integrations/images/bubble/add_action_to_workflow.png rename to sources/platform/7-integrations/images/bubble/add_action_to_workflow.png diff --git a/sources/platform/integrations/images/bubble/apify_api_token.png b/sources/platform/7-integrations/images/bubble/apify_api_token.png similarity index 100% rename from sources/platform/integrations/images/bubble/apify_api_token.png rename to sources/platform/7-integrations/images/bubble/apify_api_token.png diff --git a/sources/platform/integrations/images/bubble/button_adding_workflow.png b/sources/platform/7-integrations/images/bubble/button_adding_workflow.png similarity index 100% rename from sources/platform/integrations/images/bubble/button_adding_workflow.png rename to sources/platform/7-integrations/images/bubble/button_adding_workflow.png diff --git a/sources/platform/integrations/images/bubble/button_creating_workflow.png b/sources/platform/7-integrations/images/bubble/button_creating_workflow.png similarity index 100% rename from sources/platform/integrations/images/bubble/button_creating_workflow.png rename to sources/platform/7-integrations/images/bubble/button_creating_workflow.png diff --git a/sources/platform/integrations/images/bubble/create_workflow.png b/sources/platform/7-integrations/images/bubble/create_workflow.png similarity index 100% rename from sources/platform/integrations/images/bubble/create_workflow.png rename to sources/platform/7-integrations/images/bubble/create_workflow.png diff --git a/sources/platform/integrations/images/bubble/data_calls_preview.png b/sources/platform/7-integrations/images/bubble/data_calls_preview.png similarity index 100% rename from sources/platform/integrations/images/bubble/data_calls_preview.png rename to sources/platform/7-integrations/images/bubble/data_calls_preview.png diff --git a/sources/platform/integrations/images/bubble/data_privacy.png b/sources/platform/7-integrations/images/bubble/data_privacy.png similarity index 100% rename from sources/platform/integrations/images/bubble/data_privacy.png rename to sources/platform/7-integrations/images/bubble/data_privacy.png diff --git a/sources/platform/integrations/images/bubble/data_select_user_api_key.png b/sources/platform/7-integrations/images/bubble/data_select_user_api_key.png similarity index 100% rename from sources/platform/integrations/images/bubble/data_select_user_api_key.png rename to sources/platform/7-integrations/images/bubble/data_select_user_api_key.png diff --git a/sources/platform/integrations/images/bubble/data_type_api_key.png b/sources/platform/7-integrations/images/bubble/data_type_api_key.png similarity index 100% rename from sources/platform/integrations/images/bubble/data_type_api_key.png rename to sources/platform/7-integrations/images/bubble/data_type_api_key.png diff --git a/sources/platform/integrations/images/bubble/insert_dynamic_data.png b/sources/platform/7-integrations/images/bubble/insert_dynamic_data.png similarity index 100% rename from sources/platform/integrations/images/bubble/insert_dynamic_data.png rename to sources/platform/7-integrations/images/bubble/insert_dynamic_data.png diff --git a/sources/platform/integrations/images/bubble/plugin_actions.png b/sources/platform/7-integrations/images/bubble/plugin_actions.png similarity index 100% rename from sources/platform/integrations/images/bubble/plugin_actions.png rename to sources/platform/7-integrations/images/bubble/plugin_actions.png diff --git a/sources/platform/integrations/images/bubble/plugin_install_preview.png b/sources/platform/7-integrations/images/bubble/plugin_install_preview.png similarity index 100% rename from sources/platform/integrations/images/bubble/plugin_install_preview.png rename to sources/platform/7-integrations/images/bubble/plugin_install_preview.png diff --git a/sources/platform/integrations/images/bubble/repeating_group.png b/sources/platform/7-integrations/images/bubble/repeating_group.png similarity index 100% rename from sources/platform/integrations/images/bubble/repeating_group.png rename to sources/platform/7-integrations/images/bubble/repeating_group.png diff --git a/sources/platform/integrations/images/bubble/step1_scraping.png b/sources/platform/7-integrations/images/bubble/step1_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step1_scraping.png rename to sources/platform/7-integrations/images/bubble/step1_scraping.png diff --git a/sources/platform/integrations/images/bubble/step2_scraping.png b/sources/platform/7-integrations/images/bubble/step2_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step2_scraping.png rename to sources/platform/7-integrations/images/bubble/step2_scraping.png diff --git a/sources/platform/integrations/images/bubble/step3_scraping.png b/sources/platform/7-integrations/images/bubble/step3_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step3_scraping.png rename to sources/platform/7-integrations/images/bubble/step3_scraping.png diff --git a/sources/platform/integrations/images/bubble/step4_scraping.png b/sources/platform/7-integrations/images/bubble/step4_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step4_scraping.png rename to sources/platform/7-integrations/images/bubble/step4_scraping.png diff --git a/sources/platform/integrations/images/bubble/step5_scraping.png b/sources/platform/7-integrations/images/bubble/step5_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step5_scraping.png rename to sources/platform/7-integrations/images/bubble/step5_scraping.png diff --git a/sources/platform/integrations/images/bubble/step6_scraping.png b/sources/platform/7-integrations/images/bubble/step6_scraping.png similarity index 100% rename from sources/platform/integrations/images/bubble/step6_scraping.png rename to sources/platform/7-integrations/images/bubble/step6_scraping.png diff --git a/sources/platform/integrations/images/bubble/text_dynamic_content.png b/sources/platform/7-integrations/images/bubble/text_dynamic_content.png similarity index 100% rename from sources/platform/integrations/images/bubble/text_dynamic_content.png rename to sources/platform/7-integrations/images/bubble/text_dynamic_content.png diff --git a/sources/platform/integrations/images/bubble/user_dataset_repeating_group.png b/sources/platform/7-integrations/images/bubble/user_dataset_repeating_group.png similarity index 100% rename from sources/platform/integrations/images/bubble/user_dataset_repeating_group.png rename to sources/platform/7-integrations/images/bubble/user_dataset_repeating_group.png diff --git a/sources/platform/integrations/images/bubble/user_dataset_repeating_group_cell.png b/sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_cell.png similarity index 100% rename from sources/platform/integrations/images/bubble/user_dataset_repeating_group_cell.png rename to sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_cell.png diff --git a/sources/platform/integrations/images/bubble/user_dataset_repeating_group_set.png b/sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_set.png similarity index 100% rename from sources/platform/integrations/images/bubble/user_dataset_repeating_group_set.png rename to sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_set.png diff --git a/sources/platform/integrations/images/bubble/user_dataset_repeating_group_source.png b/sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_source.png similarity index 100% rename from sources/platform/integrations/images/bubble/user_dataset_repeating_group_source.png rename to sources/platform/7-integrations/images/bubble/user_dataset_repeating_group_source.png diff --git a/sources/platform/integrations/images/datasets-app.png b/sources/platform/7-integrations/images/datasets-app.png similarity index 100% rename from sources/platform/integrations/images/datasets-app.png rename to sources/platform/7-integrations/images/datasets-app.png diff --git a/sources/platform/integrations/images/flowise-2.png b/sources/platform/7-integrations/images/flowise-2.png similarity index 100% rename from sources/platform/integrations/images/flowise-2.png rename to sources/platform/7-integrations/images/flowise-2.png diff --git a/sources/platform/integrations/images/flowise-apify-api.png b/sources/platform/7-integrations/images/flowise-apify-api.png similarity index 100% rename from sources/platform/integrations/images/flowise-apify-api.png rename to sources/platform/7-integrations/images/flowise-apify-api.png diff --git a/sources/platform/integrations/images/flowise-apify-website-crawler.png b/sources/platform/7-integrations/images/flowise-apify-website-crawler.png similarity index 100% rename from sources/platform/integrations/images/flowise-apify-website-crawler.png rename to sources/platform/7-integrations/images/flowise-apify-website-crawler.png diff --git a/sources/platform/integrations/images/flowise-apify.png b/sources/platform/7-integrations/images/flowise-apify.png similarity index 100% rename from sources/platform/integrations/images/flowise-apify.png rename to sources/platform/7-integrations/images/flowise-apify.png diff --git a/sources/platform/integrations/images/flowise-start-screen.png b/sources/platform/7-integrations/images/flowise-start-screen.png similarity index 100% rename from sources/platform/integrations/images/flowise-start-screen.png rename to sources/platform/7-integrations/images/flowise-start-screen.png diff --git a/sources/platform/integrations/images/flowise.png b/sources/platform/7-integrations/images/flowise.png similarity index 100% rename from sources/platform/integrations/images/flowise.png rename to sources/platform/7-integrations/images/flowise.png diff --git a/sources/platform/integrations/images/gdrive/google-maps-task-add-google-drive-integration.png b/sources/platform/7-integrations/images/gdrive/google-maps-task-add-google-drive-integration.png similarity index 100% rename from sources/platform/integrations/images/gdrive/google-maps-task-add-google-drive-integration.png rename to sources/platform/7-integrations/images/gdrive/google-maps-task-add-google-drive-integration.png diff --git a/sources/platform/integrations/images/gdrive/google-maps-task-integration-format.png b/sources/platform/7-integrations/images/gdrive/google-maps-task-integration-format.png similarity index 100% rename from sources/platform/integrations/images/gdrive/google-maps-task-integration-format.png rename to sources/platform/7-integrations/images/gdrive/google-maps-task-integration-format.png diff --git a/sources/platform/integrations/images/gdrive/google-maps-task-integration-save.png b/sources/platform/7-integrations/images/gdrive/google-maps-task-integration-save.png similarity index 100% rename from sources/platform/integrations/images/gdrive/google-maps-task-integration-save.png rename to sources/platform/7-integrations/images/gdrive/google-maps-task-integration-save.png diff --git a/sources/platform/integrations/images/gdrive/google-maps-task-integration-setup.png b/sources/platform/7-integrations/images/gdrive/google-maps-task-integration-setup.png similarity index 100% rename from sources/platform/integrations/images/gdrive/google-maps-task-integration-setup.png rename to sources/platform/7-integrations/images/gdrive/google-maps-task-integration-setup.png diff --git a/sources/platform/integrations/images/gdrive/google-maps-task-integrations.png b/sources/platform/7-integrations/images/gdrive/google-maps-task-integrations.png similarity index 100% rename from sources/platform/integrations/images/gdrive/google-maps-task-integrations.png rename to sources/platform/7-integrations/images/gdrive/google-maps-task-integrations.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-configuration.png b/sources/platform/7-integrations/images/gmail/google-maps-task-configuration.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-configuration.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-configuration.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-email-filled.png b/sources/platform/7-integrations/images/gmail/google-maps-task-email-filled.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-email-filled.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-email-filled.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-email-format.png b/sources/platform/7-integrations/images/gmail/google-maps-task-email-format.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-email-format.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-email-format.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-email-save.png b/sources/platform/7-integrations/images/gmail/google-maps-task-email-save.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-email-save.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-email-save.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-email.png b/sources/platform/7-integrations/images/gmail/google-maps-task-email.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-email.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-email.png diff --git a/sources/platform/integrations/images/gmail/google-maps-task-integrations.png b/sources/platform/7-integrations/images/gmail/google-maps-task-integrations.png similarity index 100% rename from sources/platform/integrations/images/gmail/google-maps-task-integrations.png rename to sources/platform/7-integrations/images/gmail/google-maps-task-integrations.png diff --git a/sources/platform/integrations/images/google/google-integrations-accounts.png b/sources/platform/7-integrations/images/google/google-integrations-accounts.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-accounts.png rename to sources/platform/7-integrations/images/google/google-integrations-accounts.png diff --git a/sources/platform/integrations/images/google/google-integrations-add.png b/sources/platform/7-integrations/images/google/google-integrations-add.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-add.png rename to sources/platform/7-integrations/images/google/google-integrations-add.png diff --git a/sources/platform/integrations/images/google/google-integrations-connect-drive.png b/sources/platform/7-integrations/images/google/google-integrations-connect-drive.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-connect-drive.png rename to sources/platform/7-integrations/images/google/google-integrations-connect-drive.png diff --git a/sources/platform/integrations/images/google/google-integrations-connect-gmail.png b/sources/platform/7-integrations/images/google/google-integrations-connect-gmail.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-connect-gmail.png rename to sources/platform/7-integrations/images/google/google-integrations-connect-gmail.png diff --git a/sources/platform/integrations/images/google/google-integrations-details-drive.png b/sources/platform/7-integrations/images/google/google-integrations-details-drive.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-details-drive.png rename to sources/platform/7-integrations/images/google/google-integrations-details-drive.png diff --git a/sources/platform/integrations/images/google/google-integrations-details-gmail.png b/sources/platform/7-integrations/images/google/google-integrations-details-gmail.png similarity index 100% rename from sources/platform/integrations/images/google/google-integrations-details-gmail.png rename to sources/platform/7-integrations/images/google/google-integrations-details-gmail.png diff --git a/sources/platform/integrations/images/ifttt-actor-config.png b/sources/platform/7-integrations/images/ifttt-actor-config.png similarity index 100% rename from sources/platform/integrations/images/ifttt-actor-config.png rename to sources/platform/7-integrations/images/ifttt-actor-config.png diff --git a/sources/platform/integrations/images/ifttt-applet-inspect.png b/sources/platform/7-integrations/images/ifttt-applet-inspect.png similarity index 100% rename from sources/platform/integrations/images/ifttt-applet-inspect.png rename to sources/platform/7-integrations/images/ifttt-applet-inspect.png diff --git a/sources/platform/integrations/images/ifttt-applet-overview.png b/sources/platform/7-integrations/images/ifttt-applet-overview.png similarity index 100% rename from sources/platform/integrations/images/ifttt-applet-overview.png rename to sources/platform/7-integrations/images/ifttt-applet-overview.png diff --git a/sources/platform/integrations/images/ifttt-choose-action.png b/sources/platform/7-integrations/images/ifttt-choose-action.png similarity index 100% rename from sources/platform/integrations/images/ifttt-choose-action.png rename to sources/platform/7-integrations/images/ifttt-choose-action.png diff --git a/sources/platform/integrations/images/ifttt-choose-service.png b/sources/platform/7-integrations/images/ifttt-choose-service.png similarity index 100% rename from sources/platform/integrations/images/ifttt-choose-service.png rename to sources/platform/7-integrations/images/ifttt-choose-service.png diff --git a/sources/platform/integrations/images/ifttt-choose-trigger.png b/sources/platform/7-integrations/images/ifttt-choose-trigger.png similarity index 100% rename from sources/platform/integrations/images/ifttt-choose-trigger.png rename to sources/platform/7-integrations/images/ifttt-choose-trigger.png diff --git a/sources/platform/integrations/images/ifttt-connect-service.png b/sources/platform/7-integrations/images/ifttt-connect-service.png similarity index 100% rename from sources/platform/integrations/images/ifttt-connect-service.png rename to sources/platform/7-integrations/images/ifttt-connect-service.png diff --git a/sources/platform/integrations/images/integration-ready-actors.png b/sources/platform/7-integrations/images/integration-ready-actors.png similarity index 100% rename from sources/platform/integrations/images/integration-ready-actors.png rename to sources/platform/7-integrations/images/integration-ready-actors.png diff --git a/sources/platform/integrations/images/integrations-tab.png b/sources/platform/7-integrations/images/integrations-tab.png similarity index 100% rename from sources/platform/integrations/images/integrations-tab.png rename to sources/platform/7-integrations/images/integrations-tab.png diff --git a/sources/platform/integrations/images/keboola/keboola-component-setup.png b/sources/platform/7-integrations/images/keboola/keboola-component-setup.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-component-setup.png rename to sources/platform/7-integrations/images/keboola/keboola-component-setup.png diff --git a/sources/platform/integrations/images/keboola/keboola-components.png b/sources/platform/7-integrations/images/keboola/keboola-components.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-components.png rename to sources/platform/7-integrations/images/keboola/keboola-components.png diff --git a/sources/platform/integrations/images/keboola/keboola-configure-component.png b/sources/platform/7-integrations/images/keboola/keboola-configure-component.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-configure-component.png rename to sources/platform/7-integrations/images/keboola/keboola-configure-component.png diff --git a/sources/platform/integrations/images/keboola/keboola-connector.png b/sources/platform/7-integrations/images/keboola/keboola-connector.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-connector.png rename to sources/platform/7-integrations/images/keboola/keboola-connector.png diff --git a/sources/platform/integrations/images/keboola/keboola-create-configuration.png b/sources/platform/7-integrations/images/keboola/keboola-create-configuration.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-create-configuration.png rename to sources/platform/7-integrations/images/keboola/keboola-create-configuration.png diff --git a/sources/platform/integrations/images/keboola/keboola-dashboard.png b/sources/platform/7-integrations/images/keboola/keboola-dashboard.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-dashboard.png rename to sources/platform/7-integrations/images/keboola/keboola-dashboard.png diff --git a/sources/platform/integrations/images/keboola/keboola-run-component.png b/sources/platform/7-integrations/images/keboola/keboola-run-component.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-run-component.png rename to sources/platform/7-integrations/images/keboola/keboola-run-component.png diff --git a/sources/platform/integrations/images/keboola/keboola-setup-api-token.png b/sources/platform/7-integrations/images/keboola/keboola-setup-api-token.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-setup-api-token.png rename to sources/platform/7-integrations/images/keboola/keboola-setup-api-token.png diff --git a/sources/platform/integrations/images/keboola/keboola-setup-specification.png b/sources/platform/7-integrations/images/keboola/keboola-setup-specification.png similarity index 100% rename from sources/platform/integrations/images/keboola/keboola-setup-specification.png rename to sources/platform/7-integrations/images/keboola/keboola-setup-specification.png diff --git a/sources/platform/integrations/images/langflow/agent_configuration.png b/sources/platform/7-integrations/images/langflow/agent_configuration.png similarity index 100% rename from sources/platform/integrations/images/langflow/agent_configuration.png rename to sources/platform/7-integrations/images/langflow/agent_configuration.png diff --git a/sources/platform/integrations/images/langflow/agent_output.png b/sources/platform/7-integrations/images/langflow/agent_output.png similarity index 100% rename from sources/platform/integrations/images/langflow/agent_output.png rename to sources/platform/7-integrations/images/langflow/agent_output.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_configuration.png b/sources/platform/7-integrations/images/langflow/apify_actors_configuration.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_configuration.png rename to sources/platform/7-integrations/images/langflow/apify_actors_configuration.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_output.png b/sources/platform/7-integrations/images/langflow/apify_actors_output.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_output.png rename to sources/platform/7-integrations/images/langflow/apify_actors_output.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_output_data.png b/sources/platform/7-integrations/images/langflow/apify_actors_output_data.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_output_data.png rename to sources/platform/7-integrations/images/langflow/apify_actors_output_data.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_output_data_filtered.png b/sources/platform/7-integrations/images/langflow/apify_actors_output_data_filtered.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_output_data_filtered.png rename to sources/platform/7-integrations/images/langflow/apify_actors_output_data_filtered.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_output_filter.png b/sources/platform/7-integrations/images/langflow/apify_actors_output_filter.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_output_filter.png rename to sources/platform/7-integrations/images/langflow/apify_actors_output_filter.png diff --git a/sources/platform/integrations/images/langflow/apify_actors_run.png b/sources/platform/7-integrations/images/langflow/apify_actors_run.png similarity index 100% rename from sources/platform/integrations/images/langflow/apify_actors_run.png rename to sources/platform/7-integrations/images/langflow/apify_actors_run.png diff --git a/sources/platform/integrations/images/langflow/bundles_apify.png b/sources/platform/7-integrations/images/langflow/bundles_apify.png similarity index 100% rename from sources/platform/integrations/images/langflow/bundles_apify.png rename to sources/platform/7-integrations/images/langflow/bundles_apify.png diff --git a/sources/platform/integrations/images/langflow/chat_message_output.png b/sources/platform/7-integrations/images/langflow/chat_message_output.png similarity index 100% rename from sources/platform/integrations/images/langflow/chat_message_output.png rename to sources/platform/7-integrations/images/langflow/chat_message_output.png diff --git a/sources/platform/integrations/images/langflow/flow.png b/sources/platform/7-integrations/images/langflow/flow.png similarity index 100% rename from sources/platform/integrations/images/langflow/flow.png rename to sources/platform/7-integrations/images/langflow/flow.png diff --git a/sources/platform/integrations/images/langflow/new_blank_flow.png b/sources/platform/7-integrations/images/langflow/new_blank_flow.png similarity index 100% rename from sources/platform/integrations/images/langflow/new_blank_flow.png rename to sources/platform/7-integrations/images/langflow/new_blank_flow.png diff --git a/sources/platform/integrations/images/langflow/playground.png b/sources/platform/7-integrations/images/langflow/playground.png similarity index 100% rename from sources/platform/integrations/images/langflow/playground.png rename to sources/platform/7-integrations/images/langflow/playground.png diff --git a/sources/platform/integrations/images/make-com/make-com-async-1.png b/sources/platform/7-integrations/images/make-com/make-com-async-1.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-async-1.png rename to sources/platform/7-integrations/images/make-com/make-com-async-1.png diff --git a/sources/platform/integrations/images/make-com/make-com-async-2.png b/sources/platform/7-integrations/images/make-com/make-com-async-2.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-async-2.png rename to sources/platform/7-integrations/images/make-com/make-com-async-2.png diff --git a/sources/platform/integrations/images/make-com/make-com-async-3.png b/sources/platform/7-integrations/images/make-com/make-com-async-3.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-async-3.png rename to sources/platform/7-integrations/images/make-com/make-com-async-3.png diff --git a/sources/platform/integrations/images/make-com/make-com-sync-1.png b/sources/platform/7-integrations/images/make-com/make-com-sync-1.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-sync-1.png rename to sources/platform/7-integrations/images/make-com/make-com-sync-1.png diff --git a/sources/platform/integrations/images/make-com/make-com-sync-2.png b/sources/platform/7-integrations/images/make-com/make-com-sync-2.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-sync-2.png rename to sources/platform/7-integrations/images/make-com/make-com-sync-2.png diff --git a/sources/platform/integrations/images/make-com/make-com-sync-3.png b/sources/platform/7-integrations/images/make-com/make-com-sync-3.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-sync-3.png rename to sources/platform/7-integrations/images/make-com/make-com-sync-3.png diff --git a/sources/platform/integrations/images/make-com/make-com-sync-4.png b/sources/platform/7-integrations/images/make-com/make-com-sync-4.png similarity index 100% rename from sources/platform/integrations/images/make-com/make-com-sync-4.png rename to sources/platform/7-integrations/images/make-com/make-com-sync-4.png diff --git a/sources/platform/integrations/images/n8n-api-auth.png b/sources/platform/7-integrations/images/n8n-api-auth.png similarity index 100% rename from sources/platform/integrations/images/n8n-api-auth.png rename to sources/platform/7-integrations/images/n8n-api-auth.png diff --git a/sources/platform/integrations/images/n8n-get-dataset-items-example.png b/sources/platform/7-integrations/images/n8n-get-dataset-items-example.png similarity index 100% rename from sources/platform/integrations/images/n8n-get-dataset-items-example.png rename to sources/platform/7-integrations/images/n8n-get-dataset-items-example.png diff --git a/sources/platform/integrations/images/n8n-install-node-cloud.png b/sources/platform/7-integrations/images/n8n-install-node-cloud.png similarity index 100% rename from sources/platform/integrations/images/n8n-install-node-cloud.png rename to sources/platform/7-integrations/images/n8n-install-node-cloud.png diff --git a/sources/platform/integrations/images/n8n-install-node-self-hosted.png b/sources/platform/7-integrations/images/n8n-install-node-self-hosted.png similarity index 100% rename from sources/platform/integrations/images/n8n-install-node-self-hosted.png rename to sources/platform/7-integrations/images/n8n-install-node-self-hosted.png diff --git a/sources/platform/integrations/images/n8n-list-of-operations.png b/sources/platform/7-integrations/images/n8n-list-of-operations.png similarity index 100% rename from sources/platform/integrations/images/n8n-list-of-operations.png rename to sources/platform/7-integrations/images/n8n-list-of-operations.png diff --git a/sources/platform/integrations/images/n8n-oauth.png b/sources/platform/7-integrations/images/n8n-oauth.png similarity index 100% rename from sources/platform/integrations/images/n8n-oauth.png rename to sources/platform/7-integrations/images/n8n-oauth.png diff --git a/sources/platform/integrations/images/n8n-run-actor-example.png b/sources/platform/7-integrations/images/n8n-run-actor-example.png similarity index 100% rename from sources/platform/integrations/images/n8n-run-actor-example.png rename to sources/platform/7-integrations/images/n8n-run-actor-example.png diff --git a/sources/platform/integrations/images/n8n-trigger-example.png b/sources/platform/7-integrations/images/n8n-trigger-example.png similarity index 100% rename from sources/platform/integrations/images/n8n-trigger-example.png rename to sources/platform/7-integrations/images/n8n-trigger-example.png diff --git a/sources/platform/integrations/images/n8n-workflow-example.png b/sources/platform/7-integrations/images/n8n-workflow-example.png similarity index 100% rename from sources/platform/integrations/images/n8n-workflow-example.png rename to sources/platform/7-integrations/images/n8n-workflow-example.png diff --git a/sources/platform/integrations/images/openai-assistant-rag-web-browser.png b/sources/platform/7-integrations/images/openai-assistant-rag-web-browser.png similarity index 100% rename from sources/platform/integrations/images/openai-assistant-rag-web-browser.png rename to sources/platform/7-integrations/images/openai-assistant-rag-web-browser.png diff --git a/sources/platform/integrations/images/openai-vector-store-integration.png b/sources/platform/7-integrations/images/openai-vector-store-integration.png similarity index 100% rename from sources/platform/integrations/images/openai-vector-store-integration.png rename to sources/platform/7-integrations/images/openai-vector-store-integration.png diff --git a/sources/platform/integrations/images/pinecone-create-index.png b/sources/platform/7-integrations/images/pinecone-create-index.png similarity index 100% rename from sources/platform/integrations/images/pinecone-create-index.png rename to sources/platform/7-integrations/images/pinecone-create-index.png diff --git a/sources/platform/integrations/images/pinecone-integration-setup.png b/sources/platform/7-integrations/images/pinecone-integration-setup.png similarity index 100% rename from sources/platform/integrations/images/pinecone-integration-setup.png rename to sources/platform/7-integrations/images/pinecone-integration-setup.png diff --git a/sources/platform/integrations/images/pinecone-wcc-integration.png b/sources/platform/7-integrations/images/pinecone-wcc-integration.png similarity index 100% rename from sources/platform/integrations/images/pinecone-wcc-integration.png rename to sources/platform/7-integrations/images/pinecone-wcc-integration.png diff --git a/sources/platform/integrations/images/qdrant-integration-setup.png b/sources/platform/7-integrations/images/qdrant-integration-setup.png similarity index 100% rename from sources/platform/integrations/images/qdrant-integration-setup.png rename to sources/platform/7-integrations/images/qdrant-integration-setup.png diff --git a/sources/platform/integrations/images/qdrant-wcc-integration.png b/sources/platform/7-integrations/images/qdrant-wcc-integration.png similarity index 100% rename from sources/platform/integrations/images/qdrant-wcc-integration.png rename to sources/platform/7-integrations/images/qdrant-wcc-integration.png diff --git a/sources/platform/integrations/images/slack-apify-message.png b/sources/platform/7-integrations/images/slack-apify-message.png similarity index 100% rename from sources/platform/integrations/images/slack-apify-message.png rename to sources/platform/7-integrations/images/slack-apify-message.png diff --git a/sources/platform/integrations/images/slack-integration-setup.png b/sources/platform/7-integrations/images/slack-integration-setup.png similarity index 100% rename from sources/platform/integrations/images/slack-integration-setup.png rename to sources/platform/7-integrations/images/slack-integration-setup.png diff --git a/sources/platform/integrations/images/zapier-action-1.png b/sources/platform/7-integrations/images/zapier-action-1.png similarity index 100% rename from sources/platform/integrations/images/zapier-action-1.png rename to sources/platform/7-integrations/images/zapier-action-1.png diff --git a/sources/platform/integrations/images/zapier-action-2.png b/sources/platform/7-integrations/images/zapier-action-2.png similarity index 100% rename from sources/platform/integrations/images/zapier-action-2.png rename to sources/platform/7-integrations/images/zapier-action-2.png diff --git a/sources/platform/integrations/images/zapier-action-3.png b/sources/platform/7-integrations/images/zapier-action-3.png similarity index 100% rename from sources/platform/integrations/images/zapier-action-3.png rename to sources/platform/7-integrations/images/zapier-action-3.png diff --git a/sources/platform/integrations/images/zapier-allow-access.png b/sources/platform/7-integrations/images/zapier-allow-access.png similarity index 100% rename from sources/platform/integrations/images/zapier-allow-access.png rename to sources/platform/7-integrations/images/zapier-allow-access.png diff --git a/sources/platform/integrations/images/zapier-choose-account.png b/sources/platform/7-integrations/images/zapier-choose-account.png similarity index 100% rename from sources/platform/integrations/images/zapier-choose-account.png rename to sources/platform/7-integrations/images/zapier-choose-account.png diff --git a/sources/platform/integrations/images/zapier-dashboard-zaps.png b/sources/platform/7-integrations/images/zapier-dashboard-zaps.png similarity index 100% rename from sources/platform/integrations/images/zapier-dashboard-zaps.png rename to sources/platform/7-integrations/images/zapier-dashboard-zaps.png diff --git a/sources/platform/integrations/images/zapier-new-connection.png b/sources/platform/7-integrations/images/zapier-new-connection.png similarity index 100% rename from sources/platform/integrations/images/zapier-new-connection.png rename to sources/platform/7-integrations/images/zapier-new-connection.png diff --git a/sources/platform/integrations/images/zapier-trigger-2.png b/sources/platform/7-integrations/images/zapier-trigger-2.png similarity index 100% rename from sources/platform/integrations/images/zapier-trigger-2.png rename to sources/platform/7-integrations/images/zapier-trigger-2.png diff --git a/sources/platform/integrations/images/zapier-trigger.png b/sources/platform/7-integrations/images/zapier-trigger.png similarity index 100% rename from sources/platform/integrations/images/zapier-trigger.png rename to sources/platform/7-integrations/images/zapier-trigger.png diff --git a/sources/platform/integrations/images/zapier-zap-action.png b/sources/platform/7-integrations/images/zapier-zap-action.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-action.png rename to sources/platform/7-integrations/images/zapier-zap-action.png diff --git a/sources/platform/integrations/images/zapier-zap-choose-action.png b/sources/platform/7-integrations/images/zapier-zap-choose-action.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-choose-action.png rename to sources/platform/7-integrations/images/zapier-zap-choose-action.png diff --git a/sources/platform/integrations/images/zapier-zap-choose-event-2.png b/sources/platform/7-integrations/images/zapier-zap-choose-event-2.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-choose-event-2.png rename to sources/platform/7-integrations/images/zapier-zap-choose-event-2.png diff --git a/sources/platform/integrations/images/zapier-zap-choose-event.png b/sources/platform/7-integrations/images/zapier-zap-choose-event.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-choose-event.png rename to sources/platform/7-integrations/images/zapier-zap-choose-event.png diff --git a/sources/platform/integrations/images/zapier-zap-telegram-access.png b/sources/platform/7-integrations/images/zapier-zap-telegram-access.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-telegram-access.png rename to sources/platform/7-integrations/images/zapier-zap-telegram-access.png diff --git a/sources/platform/integrations/images/zapier-zap-trigger-choose.png b/sources/platform/7-integrations/images/zapier-zap-trigger-choose.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-trigger-choose.png rename to sources/platform/7-integrations/images/zapier-zap-trigger-choose.png diff --git a/sources/platform/integrations/images/zapier-zap-trigger-chosen.png b/sources/platform/7-integrations/images/zapier-zap-trigger-chosen.png similarity index 100% rename from sources/platform/integrations/images/zapier-zap-trigger-chosen.png rename to sources/platform/7-integrations/images/zapier-zap-trigger-chosen.png diff --git a/sources/platform/integrations/index.mdx b/sources/platform/7-integrations/index.mdx similarity index 87% rename from sources/platform/integrations/index.mdx rename to sources/platform/7-integrations/index.mdx index 72fc3976d3..c4422ade4d 100644 --- a/sources/platform/integrations/index.mdx +++ b/sources/platform/7-integrations/index.mdx @@ -1,8 +1,7 @@ --- -title: Integrations -description: Learn how to integrate the Apify platform with other services, your systems, data pipelines, and other web automation workflows. -sidebar_position: 11 -category: platform +title: "Integrations" +description: "Connect Apify with external services" +sidebar_position: 1 slug: /integrations --- @@ -19,7 +18,7 @@ import CardGrid from '@site/src/components/CardGrid'; Integrations allow you to combine separate applications and take advantage of their combined capabilities. Automation of these online processes increases your productivity. That's why we made Apify in a way that allows you to connect it with practically any cloud service or web app and make it part of your larger projects. -If you are building a service and your users could benefit from integrating with Apify or wise-versa then ready the [integrate with Apify](./integrate_with_apify.md). +If you are building a service and your users could benefit from integrating with Apify or wise-versa then ready the [integrate with Apify](/platform/integrations/integrate-with-apify). ## Built-in integrations @@ -106,25 +105,25 @@ If you use one of the main integration platforms, Apify's support is here for yo @@ -149,7 +148,7 @@ The Apify platform integrates with popular ETL and data pipeline services, enabl @@ -168,92 +167,92 @@ These integrations allow you to use Apify Actors as tools and data sources. -_API token_ - Apify user generates personal API token from Apify account settings page. For more information, see [API Token documentation](https://docs.apify.com/platform/integrations/api#api-token). +_API token_ - Apify user generates personal API token from Apify account settings page. For more information, see [API Token documentation](https://docs.apify.com///platform/integrations/api#api-token). ### API implementation @@ -77,7 +77,7 @@ Recommended features: - Select Actor: The Actor list will be pre-populated with Actors that the user created or used, using the [List of Actors API](https://docs.apify.com/api/v2/acts-get) and enriched with Actors from the store, which the user has not run already using [API](https://docs.apify.com/api/v2/store-get). - Synchronous vs. asynchronous run: flow will wait until the run/task finishes (consider a timeout on your platform side) - Input UI: upon selecting an Actor, dynamically display specific Actor input and preload default example values based on the Actor Input schema. Alternatively, allow users to insert a JSON input for the Actor. -- Additionally, it should include the option to choose [Actor build](https://docs.apify.com/platform/actors/running/runs-and-builds), [memory](https://docs.apify.com/platform/actors/running/usage-and-resources#memory), and [timeout](https://docs.apify.com/platform/actors/running/usage-and-resources#memory). +- Additionally, it should include the option to choose [Actor build](https://docs.apify.com/platform/using-actors/running/runs-and-builds), [memory](https://docs.apify.com/platform/using-actors/running/usage-and-resources#memory), and [timeout](https://docs.apify.com/platform/using-actors/running/usage-and-resources#memory). - Field mapping: allowing users to map fields to data acquired in previous steps of the workflow. ##### Run a task diff --git a/sources/platform/integrations/programming/_category_.yml b/sources/platform/7-integrations/programming/_category_.yml similarity index 100% rename from sources/platform/integrations/programming/_category_.yml rename to sources/platform/7-integrations/programming/_category_.yml diff --git a/sources/platform/integrations/programming/api.md b/sources/platform/7-integrations/programming/api.md similarity index 97% rename from sources/platform/integrations/programming/api.md rename to sources/platform/7-integrations/programming/api.md index b303a0ffa4..d221e57709 100644 --- a/sources/platform/integrations/programming/api.md +++ b/sources/platform/7-integrations/programming/api.md @@ -158,7 +158,7 @@ When you run an Actor with a scoped token in this mode, Apify will inject a toke This way you can be sure that Actors won't accidentally—or intentionally—access any data they shouldn't. However, Actors might not function properly if the scope is not sufficient. :::caution -Restricted access mode is not supported for Actors running in [Standby mode](/platform/actors/running/standby). While you can send standby requests using a scoped token configured with restricted access, functionality is not guaranteed. +Restricted access mode is not supported for Actors running in [Standby mode](/platform/using-actors/running/standby). While you can send standby requests using a scoped token configured with restricted access, functionality is not guaranteed. :::tip This restriction is _transitive_, which means that if the Actor runs another Actor, its access will be restricted as well. @@ -166,7 +166,7 @@ This restriction is _transitive_, which means that if the Actor runs another Act #### Default run storages -When Apify [runs an Actor](/platform/actors/running/runs-and-builds#runs), it automatically creates a set of default storages (a dataset, a key-value store and request queue) that the Actor can use in runtime. +When Apify [runs an Actor](/platform/using-actors/running/runs-and-builds#runs), it automatically creates a set of default storages (a dataset, a key-value store and request queue) that the Actor can use in runtime. You can configure whether the scoped token you are going use to run the Actor should get **Write** access to these default storages. diff --git a/sources/platform/integrations/programming/github.md b/sources/platform/7-integrations/programming/github.md similarity index 100% rename from sources/platform/integrations/programming/github.md rename to sources/platform/7-integrations/programming/github.md diff --git a/sources/platform/integrations/programming/webhooks/actions.md b/sources/platform/7-integrations/programming/webhooks/actions.md similarity index 100% rename from sources/platform/integrations/programming/webhooks/actions.md rename to sources/platform/7-integrations/programming/webhooks/actions.md diff --git a/sources/platform/integrations/programming/webhooks/ad_hoc_webhooks.md b/sources/platform/7-integrations/programming/webhooks/ad_hoc_webhooks.md similarity index 100% rename from sources/platform/integrations/programming/webhooks/ad_hoc_webhooks.md rename to sources/platform/7-integrations/programming/webhooks/ad_hoc_webhooks.md diff --git a/sources/platform/integrations/programming/webhooks/events.md b/sources/platform/7-integrations/programming/webhooks/events.md similarity index 100% rename from sources/platform/integrations/programming/webhooks/events.md rename to sources/platform/7-integrations/programming/webhooks/events.md diff --git a/sources/platform/integrations/programming/webhooks/index.md b/sources/platform/7-integrations/programming/webhooks/index.md similarity index 100% rename from sources/platform/integrations/programming/webhooks/index.md rename to sources/platform/7-integrations/programming/webhooks/index.md diff --git a/sources/platform/integrations/workflows-and-notifications/_category_.yml b/sources/platform/7-integrations/workflows-and-notifications/_category_.yml similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/_category_.yml rename to sources/platform/7-integrations/workflows-and-notifications/_category_.yml diff --git a/sources/platform/integrations/workflows-and-notifications/bubble.md b/sources/platform/7-integrations/workflows-and-notifications/bubble.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/bubble.md rename to sources/platform/7-integrations/workflows-and-notifications/bubble.md diff --git a/sources/platform/integrations/workflows-and-notifications/gmail.md b/sources/platform/7-integrations/workflows-and-notifications/gmail.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gmail.md rename to sources/platform/7-integrations/workflows-and-notifications/gmail.md diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_credential_gumloop.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_credential_gumloop.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_credential_gumloop.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_credential_gumloop.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node_library.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node_library.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node_library.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/apify_task_runner_node_library.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/create_apify_task.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/create_apify_task.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/create_apify_task.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/create_apify_task.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/generate_apify_token.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/generate_apify_token.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/generate_apify_token.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/generate_apify_token.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/input-outputs.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/input-outputs.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/input-outputs.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/input-outputs.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/mcp-node-image.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/mcp-node-image.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/mcp-node-image.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/mcp-node-image.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/node.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/node.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/node.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/node.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/prompt.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/prompt.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/instagram/prompt.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/instagram/prompt.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/input-outputs.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/input-outputs.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/input-outputs.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/input-outputs.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/mcp-node-image.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/mcp-node-image.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/mcp-node-image.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/mcp-node-image.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/node.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/node.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/node.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/node.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/prompt.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/prompt.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/maps/prompt.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/maps/prompt.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/start_apify_task.png b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/start_apify_task.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/start_apify_task.png rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/start_apify_task.png diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/input-output.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/input-output.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/input-output.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/input-output.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/mcp-node-image.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/mcp-node-image.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/mcp-node-image.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/mcp-node-image.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/node.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/node.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/node.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/node.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/prompt.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/prompt.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/tiktok/prompt.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/tiktok/prompt.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/input-outputs.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/input-outputs.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/input-outputs.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/input-outputs.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/mcp-node-image.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/mcp-node-image.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/mcp-node-image.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/mcp-node-image.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/node.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/node.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/node.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/node.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/prompt.jpeg b/sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/prompt.jpeg similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/images/youtube/prompt.jpeg rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/images/youtube/prompt.jpeg diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/index.md b/sources/platform/7-integrations/workflows-and-notifications/gumloop/index.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/index.md rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/index.md diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/instagram.md b/sources/platform/7-integrations/workflows-and-notifications/gumloop/instagram.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/instagram.md rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/instagram.md diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/maps.md b/sources/platform/7-integrations/workflows-and-notifications/gumloop/maps.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/maps.md rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/maps.md diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/tiktok.md b/sources/platform/7-integrations/workflows-and-notifications/gumloop/tiktok.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/tiktok.md rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/tiktok.md diff --git a/sources/platform/integrations/workflows-and-notifications/gumloop/youtube.md b/sources/platform/7-integrations/workflows-and-notifications/gumloop/youtube.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/gumloop/youtube.md rename to sources/platform/7-integrations/workflows-and-notifications/gumloop/youtube.md diff --git a/sources/platform/integrations/workflows-and-notifications/ifttt.md b/sources/platform/7-integrations/workflows-and-notifications/ifttt.md similarity index 93% rename from sources/platform/integrations/workflows-and-notifications/ifttt.md rename to sources/platform/7-integrations/workflows-and-notifications/ifttt.md index 0c700d8565..7ce68e038a 100644 --- a/sources/platform/integrations/workflows-and-notifications/ifttt.md +++ b/sources/platform/7-integrations/workflows-and-notifications/ifttt.md @@ -93,8 +93,8 @@ To use Apify as an action in your Applet: |-----------|-------------|----------------| | **Wait until run finishes** | Defines how the Actor should be executed. | `yes`, `no` | | **Input overrides** | JSON input that overrides the Actor's default input. | `{"key": "value"}` | - | **Build** | Specifies the Actor build to run. Can be a build tag or build number. See [Builds](/platform/actors/running/runs-and-builds#builds) for more information. | `0.2.10`, `version-0` | - | **Memory** | Memory limit for the run in megabytes. See [Memory](/platform/actors/running/usage-and-resources#memory) for more information. | `256` | + | **Build** | Specifies the Actor build to run. Can be a build tag or build number. See [Builds](/platform/using-actors/running/runs-and-builds#builds) for more information. | `0.2.10`, `version-0` | + | **Memory** | Memory limit for the run in megabytes. See [Memory](/platform/using-actors/running/usage-and-resources#memory) for more information. | `256` | 1. Click **Create action** to finish setting up the action. @@ -125,9 +125,9 @@ To check if your Applet is working properly: ### Queries -- **Get Dataset Items**: Retrieves items from a [dataset](/platform/storage/dataset) +- **Get Dataset Items**: Retrieves items from a [dataset](/platform/core-concepts/storage/dataset) - **Scrape Single URL**: Runs a scraper for a specified website and returns its content -- **Get Key-Value Store Record**: Retrieves a value from a [key-value store](/platform/storage/key-value-store) +- **Get Key-Value Store Record**: Retrieves a value from a [key-value store](/platform/core-concepts/storage/key-value-store) ## Troubleshooting diff --git a/sources/platform/integrations/workflows-and-notifications/make/ai-crawling.md b/sources/platform/7-integrations/workflows-and-notifications/make/ai-crawling.md similarity index 96% rename from sources/platform/integrations/workflows-and-notifications/make/ai-crawling.md rename to sources/platform/7-integrations/workflows-and-notifications/make/ai-crawling.md index c115ca7e44..4ea9dc2747 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/ai-crawling.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/ai-crawling.md @@ -11,7 +11,7 @@ toc_max_heading_level: 4 Apify Scraper for AI Crawling from [Apify](https://apify.com/) lets you extract text content from websites to feed AI models, LLM applications, vector databases, or Retrieval Augmented Generation (RAG) pipelines. It supports rich formatting using Markdown, cleans the HTML of irrelevant elements, downloads linked files, and integrates with AI ecosystems like LangChain, LlamaIndex, and other LLM frameworks. -To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate content extraction at scale and incorporate the results into your AI workflows. +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate content extraction at scale and incorporate the results into your AI workflows. ## Connect Apify Scraper for AI Crawling diff --git a/sources/platform/integrations/workflows-and-notifications/make/amazon.md b/sources/platform/7-integrations/workflows-and-notifications/make/amazon.md similarity index 98% rename from sources/platform/integrations/workflows-and-notifications/make/amazon.md rename to sources/platform/7-integrations/workflows-and-notifications/make/amazon.md index 611999efce..6cf8b6e398 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/amazon.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/amazon.md @@ -11,7 +11,7 @@ unlisted: true The Amazon Scraper module from [Apify](https://apify.com) allows you to extract product, search, or category data from Amazon. -To use the module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use the module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. ## Connect Apify Scraper for Amazon Data modules to Make diff --git a/sources/platform/integrations/workflows-and-notifications/make/facebook.md b/sources/platform/7-integrations/workflows-and-notifications/make/facebook.md similarity index 97% rename from sources/platform/integrations/workflows-and-notifications/make/facebook.md rename to sources/platform/7-integrations/workflows-and-notifications/make/facebook.md index ecc3978a25..b8937151c3 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/facebook.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/facebook.md @@ -11,7 +11,7 @@ unlisted: true The Facebook Scraper modules from [Apify](https://apify.com/) allow you to extract posts, comments, and profile data from Facebook. -To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. ## Connect Apify Scraper for Facebook Data modules to Make diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/Apify_Console_token_for_Make.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/Apify_Console_token_for_Make.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/Apify_Console_token_for_Make.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/Apify_Console_token_for_Make.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/Apify_token_on_Make.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/Apify_token_on_Make.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/Apify_token_on_Make.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/Apify_token_on_Make.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/image 1.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/image 1.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/image 1.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/image 1.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/wcc-signup.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/wcc-signup.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/ai-crawling/wcc-signup.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/ai-crawling/wcc-signup.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/amazon/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/amazon/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/amazon/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/amazon/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/amazon/image1.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/amazon/image1.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/amazon/image1.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/amazon/image1.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/facebook/actor-rental.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/actor-rental.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/facebook/actor-rental.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/actor-rental.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/facebook/make-api-token.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/make-api-token.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/facebook/make-api-token.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/make-api-token.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/facebook/signup.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/signup.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/facebook/signup.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/signup.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/facebook/start-rental.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/start-rental.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/facebook/start-rental.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/facebook/start-rental.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/instagram/Apify_Make_Sign_up_page.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/instagram/Apify_Make_Sign_up_page.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/instagram/Apify_Make_Sign_up_page.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/instagram/Apify_Make_Sign_up_page.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/instagram/Apify_Token_for_modules_on_Make.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/instagram/Apify_Token_for_modules_on_Make.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/instagram/Apify_Token_for_modules_on_Make.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/instagram/Apify_Token_for_modules_on_Make.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/llm/apify-token-for-module-on-make.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/llm/apify-token-for-module-on-make.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/llm/apify-token-for-module-on-make.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/llm/apify-token-for-module-on-make.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/llm/rag-signup.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/llm/rag-signup.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/llm/rag-signup.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/llm/rag-signup.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/maps/Screenshot_2025-04-22_at_12.45.33.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/maps/Screenshot_2025-04-22_at_12.45.33.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/maps/Screenshot_2025-04-22_at_12.45.33.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/maps/Screenshot_2025-04-22_at_12.45.33.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/maps/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/maps/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/maps/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/maps/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/maps/maps-signup.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/maps/maps-signup.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/maps/maps-signup.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/maps/maps-signup.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/search/image 1.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/search/image 1.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/search/image 1.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/search/image 1.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/search/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/search/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/search/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/search/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/search/search-signup.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/search/search-signup.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/search/search-signup.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/search/search-signup.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/tiktok/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/tiktok/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/tiktok/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/tiktok/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/tiktok/image1.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/tiktok/image1.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/tiktok/image1.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/tiktok/image1.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/youtube/image.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/youtube/image.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/youtube/image.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/youtube/image.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/images/youtube/image1.png b/sources/platform/7-integrations/workflows-and-notifications/make/images/youtube/image1.png similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/make/images/youtube/image1.png rename to sources/platform/7-integrations/workflows-and-notifications/make/images/youtube/image1.png diff --git a/sources/platform/integrations/workflows-and-notifications/make/index.md b/sources/platform/7-integrations/workflows-and-notifications/make/index.md similarity index 99% rename from sources/platform/integrations/workflows-and-notifications/make/index.md rename to sources/platform/7-integrations/workflows-and-notifications/make/index.md index 5943aca327..6998b4f90e 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/index.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/index.md @@ -143,4 +143,4 @@ You can initiate the Actor run via the Apify console, a scheduler, or from anoth ### Searches -- **Get Dataset Items:** Retrieves items from a [dataset](/platform/storage/dataset). +- **Get Dataset Items:** Retrieves items from a [dataset](/platform/core-concepts/storage/dataset). diff --git a/sources/platform/integrations/workflows-and-notifications/make/instagram.md b/sources/platform/7-integrations/workflows-and-notifications/make/instagram.md similarity index 98% rename from sources/platform/integrations/workflows-and-notifications/make/instagram.md rename to sources/platform/7-integrations/workflows-and-notifications/make/instagram.md index 428188158b..2c1f39bbc7 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/instagram.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/instagram.md @@ -15,7 +15,7 @@ unlisted: true The Instagram Scraper modules from [Apify](https://apify.com) allow you to extract posts, comments, and profile data from Instagram. -To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. ## Connect Apify Scraper for Instagram Data modules to Make diff --git a/sources/platform/integrations/workflows-and-notifications/make/llm.md b/sources/platform/7-integrations/workflows-and-notifications/make/llm.md similarity index 96% rename from sources/platform/integrations/workflows-and-notifications/make/llm.md rename to sources/platform/7-integrations/workflows-and-notifications/make/llm.md index 1429f2c6cf..7633386610 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/llm.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/llm.md @@ -11,7 +11,7 @@ toc_max_heading_level: 4 Apify Scraper for LLMs from [Apify](https://apify.com) is a web browsing module for OpenAI Assistants, RAG pipelines, and AI agents. It can query Google Search, scrape the top results, and return page content as Markdown for downstream AI processing. -To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the Apify Console under **Settings > Integrations**. After connecting, you can automate content extraction and integrate results into your AI workflows. +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token). You can find your token in the Apify Console under **Settings > Integrations**. After connecting, you can automate content extraction and integrate results into your AI workflows. ## Connect Apify Scraper for LLMs diff --git a/sources/platform/integrations/workflows-and-notifications/make/maps.md b/sources/platform/7-integrations/workflows-and-notifications/make/maps.md similarity index 98% rename from sources/platform/integrations/workflows-and-notifications/make/maps.md rename to sources/platform/7-integrations/workflows-and-notifications/make/maps.md index d7c845c457..3a22f92708 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/maps.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/maps.md @@ -12,7 +12,7 @@ unlisted: true The Google Maps Leads Scraper modules from [apify.com](http://apify.com/) allow you to extract valuable business lead data from Google Maps, including contact information, email addresses, social media profiles, business websites, phone numbers, and detailed location data. Perfect for sales teams, marketers, and business developers looking to build targeted lead lists, marketers or other commercial teams looking to data mine reviews or assess sentiment analysis wide geographies. -To use these modules, you need an [Apify account](https://console.apify.com/sign-up) and an [API token](https://docs.apify.com/platform/integrations/api#api-token), which you can find under **Settings > Integrations** in Apify Console. After connecting, you can automate lead generation at scale and incorporate the results into your sales and marketing workflows. +To use these modules, you need an [Apify account](https://console.apify.com/sign-up) and an [API token](https://docs.apify.com///platform/integrations/api#api-token), which you can find under **Settings > Integrations** in Apify Console. After connecting, you can automate lead generation at scale and incorporate the results into your sales and marketing workflows. For more details, follow the tutorial below. diff --git a/sources/platform/integrations/workflows-and-notifications/make/search.md b/sources/platform/7-integrations/workflows-and-notifications/make/search.md similarity index 95% rename from sources/platform/integrations/workflows-and-notifications/make/search.md rename to sources/platform/7-integrations/workflows-and-notifications/make/search.md index 21d17a77ac..8bcf18e102 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/search.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/search.md @@ -11,7 +11,7 @@ unlisted: true The Google search modules from [Apify](https://apify.com) allows you to crawl Google Search Results Pages (SERPs) and extract data from those web pages in structured format such as JSON, XML, CSV, or Excel. -To use the module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use the module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. ## Connect Apify Scraper for Google Search modules to Make diff --git a/sources/platform/integrations/workflows-and-notifications/make/tiktok.md b/sources/platform/7-integrations/workflows-and-notifications/make/tiktok.md similarity index 96% rename from sources/platform/integrations/workflows-and-notifications/make/tiktok.md rename to sources/platform/7-integrations/workflows-and-notifications/make/tiktok.md index dad6f84fe2..a323406bbc 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/tiktok.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/tiktok.md @@ -11,7 +11,7 @@ unlisted: true The TikTok Scraper modules from [Apify](https://apify.com) allow you to extract hashtag, comments, and profile data from TikTok. -To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use these modules, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token). You can find your token in the [Apify Console](https://console.apify.com/) under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. ## Connect Apify Scraper for TikTok Data modules to Make diff --git a/sources/platform/integrations/workflows-and-notifications/make/youtube.md b/sources/platform/7-integrations/workflows-and-notifications/make/youtube.md similarity index 97% rename from sources/platform/integrations/workflows-and-notifications/make/youtube.md rename to sources/platform/7-integrations/workflows-and-notifications/make/youtube.md index b8af1a02cb..bce90c35bc 100644 --- a/sources/platform/integrations/workflows-and-notifications/make/youtube.md +++ b/sources/platform/7-integrations/workflows-and-notifications/make/youtube.md @@ -11,7 +11,7 @@ unlisted: true The YouTube Scraper module from [apify.com](https://apify.com) allows you to extract channel, video, streams, shorts, and search data from YouTube. -To use this module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com/platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. +To use this module, you need an [Apify account](https://console.apify.com) and an [API token](https://docs.apify.com///platform/integrations/api#api-token), which you can find in the Apify Console under **Settings > Integrations**. After connecting, you can automate data extraction and incorporate the results into your workflows. For more details, follow the tutorial below. diff --git a/sources/platform/integrations/workflows-and-notifications/n8n.md b/sources/platform/7-integrations/workflows-and-notifications/n8n.md similarity index 96% rename from sources/platform/integrations/workflows-and-notifications/n8n.md rename to sources/platform/7-integrations/workflows-and-notifications/n8n.md index 9e27a19e69..37fd7133e8 100644 --- a/sources/platform/integrations/workflows-and-notifications/n8n.md +++ b/sources/platform/7-integrations/workflows-and-notifications/n8n.md @@ -117,7 +117,7 @@ Actions allow you to perform operations like running an Actor within a workflow. 1. Click **Add Node**, search for **Apify**, and select it. 1. Select any operation. In this example we will use **Run Actor**. 1. Configure it: - - **Custom input**: JSON input for the Actor run, which you can find on the Actor input page in Apify Console. See [Inputs](/platform/actors/running/input-and-output#input) for more information. If empty, the run uses the input specified in the default run configuration + - **Custom input**: JSON input for the Actor run, which you can find on the Actor input page in Apify Console. See [Inputs](/platform/using-actors/running/input-and-output#input) for more information. If empty, the run uses the input specified in the default run configuration - **Timeout**: Timeout for the Actor run in seconds. Zero value means there is no timeout - **Memory**: Amount of memory allocated for the Actor run, in megabytes - **Build Tag**: Specifies the Actor build tag to run. By default, the run uses the build specified in the default run configuration for the Actor (typically `latest`) @@ -162,11 +162,11 @@ Pull data from Apify storage. #### Datasets -- **Get Items**: Retrieves items from a [dataset](/platform/storage/dataset) +- **Get Items**: Retrieves items from a [dataset](/platform/core-concepts/storage/dataset) #### Key-Value Stores -- **Get Record**: Retrieves a value from a [key-value store](/platform/storage/key-value-store) +- **Get Record**: Retrieves a value from a [key-value store](/platform/core-concepts/storage/key-value-store) ### Triggers diff --git a/sources/platform/integrations/workflows-and-notifications/slack.md b/sources/platform/7-integrations/workflows-and-notifications/slack.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/slack.md rename to sources/platform/7-integrations/workflows-and-notifications/slack.md diff --git a/sources/platform/integrations/workflows-and-notifications/telegram.md b/sources/platform/7-integrations/workflows-and-notifications/telegram.md similarity index 100% rename from sources/platform/integrations/workflows-and-notifications/telegram.md rename to sources/platform/7-integrations/workflows-and-notifications/telegram.md diff --git a/sources/platform/integrations/workflows-and-notifications/zapier.md b/sources/platform/7-integrations/workflows-and-notifications/zapier.md similarity index 95% rename from sources/platform/integrations/workflows-and-notifications/zapier.md rename to sources/platform/7-integrations/workflows-and-notifications/zapier.md index b72c572746..a4f548a18a 100644 --- a/sources/platform/integrations/workflows-and-notifications/zapier.md +++ b/sources/platform/7-integrations/workflows-and-notifications/zapier.md @@ -119,13 +119,13 @@ Once you are happy with the test, you can publish the Zap. When it is turned on, ### Set Key-Value Store Record -> Sets a value to a [Key-value store](/platform/storage/key-value-store). +> Sets a value to a [Key-value store](/platform/core-concepts/storage/key-value-store). ## Searches ### Fetch Dataset Items -> Retrieves items from a [dataset](/platform/storage/dataset). +> Retrieves items from a [dataset](/platform/core-concepts/storage/dataset). ### Find Last Actor Run @@ -137,6 +137,6 @@ Once you are happy with the test, you can publish the Zap. When it is turned on, ### Get Key-Value Store Record -> Retrieves value from a [Key-value store](/platform/storage/key-value-store). +> Retrieves value from a [Key-value store](/platform/core-concepts/storage/key-value-store). If you have any questions or need help, feel free to reach out to us on our [developer community on Discord](https://discord.com/invite/jyEM2PRvMU). diff --git a/sources/platform/8-collaboration/_category_.yml b/sources/platform/8-collaboration/_category_.yml new file mode 100644 index 0000000000..78cb7fd18e --- /dev/null +++ b/sources/platform/8-collaboration/_category_.yml @@ -0,0 +1,2 @@ +label: 'Collaboration' +position: 8 \ No newline at end of file diff --git a/sources/platform/collaboration/access_rights.md b/sources/platform/8-collaboration/access_rights.md similarity index 75% rename from sources/platform/collaboration/access_rights.md rename to sources/platform/8-collaboration/access_rights.md index 219c9b43cd..5bf48b1a9f 100644 --- a/sources/platform/collaboration/access_rights.md +++ b/sources/platform/8-collaboration/access_rights.md @@ -10,10 +10,10 @@ slug: /collaboration/access-rights --- -You can easily and securely share your own resources - Actors, tasks, key-value stores, datasets, and request queues - with other users by using a [granular](https://www.google.com/search?q=define+granular+permissions) permissions system. This enables you, for example, to let your colleague run an [Actor](../actors/index.mdx) or view a [dataset](../storage/dataset.md) but not modify it. +You can easily and securely share your own resources - Actors, tasks, key-value stores, datasets, and request queues - with other users by using a [granular](https://www.google.com/search?q=define+granular+permissions) permissions system. This enables you, for example, to let your colleague run an [Actor](/platform/using-actors) or view a [dataset](/platform/core-concepts/storage/dataset) but not modify it. You can also grant permission to update an Actor and build a new version. -[Storages](../storage/index.md) (key-value stores, request queues, and datasets) are sharable in the same way as a **read** permission or a combination of both **read** and **write** permissions. +[Storages](/platform/core-concepts/storage) (key-value stores, request queues, and datasets) are sharable in the same way as a **read** permission or a combination of both **read** and **write** permissions. To share an Actor, task, key-value store, request queue, or dataset, click on the **Actions** button in the top-right corner and select **Share**. You can add a collaborator by using their **user ID**, **email**, or **username**. Once you have added the collaborator, you can configure their permissions. diff --git a/sources/platform/collaboration/general-resource-access.md b/sources/platform/8-collaboration/general-resource-access.md similarity index 100% rename from sources/platform/collaboration/general-resource-access.md rename to sources/platform/8-collaboration/general-resource-access.md diff --git a/sources/platform/collaboration/images/access-rights/access-rights.png b/sources/platform/8-collaboration/images/access-rights/access-rights.png similarity index 100% rename from sources/platform/collaboration/images/access-rights/access-rights.png rename to sources/platform/8-collaboration/images/access-rights/access-rights.png diff --git a/sources/platform/collaboration/images/general-resouce-access/account-setting.png b/sources/platform/8-collaboration/images/general-resouce-access/account-setting.png similarity index 100% rename from sources/platform/collaboration/images/general-resouce-access/account-setting.png rename to sources/platform/8-collaboration/images/general-resouce-access/account-setting.png diff --git a/sources/platform/collaboration/images/general-resouce-access/copy-shareable-link.png b/sources/platform/8-collaboration/images/general-resouce-access/copy-shareable-link.png similarity index 100% rename from sources/platform/collaboration/images/general-resouce-access/copy-shareable-link.png rename to sources/platform/8-collaboration/images/general-resouce-access/copy-shareable-link.png diff --git a/sources/platform/collaboration/images/general-resouce-access/creating-actor-issue.png b/sources/platform/8-collaboration/images/general-resouce-access/creating-actor-issue.png similarity index 100% rename from sources/platform/collaboration/images/general-resouce-access/creating-actor-issue.png rename to sources/platform/8-collaboration/images/general-resouce-access/creating-actor-issue.png diff --git a/sources/platform/collaboration/images/general-resouce-access/share-resource-dialog.png b/sources/platform/8-collaboration/images/general-resouce-access/share-resource-dialog.png similarity index 100% rename from sources/platform/collaboration/images/general-resouce-access/share-resource-dialog.png rename to sources/platform/8-collaboration/images/general-resouce-access/share-resource-dialog.png diff --git a/sources/platform/collaboration/images/organizations/configure-permissions.png b/sources/platform/8-collaboration/images/organizations/configure-permissions.png similarity index 100% rename from sources/platform/collaboration/images/organizations/configure-permissions.png rename to sources/platform/8-collaboration/images/organizations/configure-permissions.png diff --git a/sources/platform/collaboration/images/organizations/convert-to-organization.png b/sources/platform/8-collaboration/images/organizations/convert-to-organization.png similarity index 100% rename from sources/platform/collaboration/images/organizations/convert-to-organization.png rename to sources/platform/8-collaboration/images/organizations/convert-to-organization.png diff --git a/sources/platform/collaboration/images/organizations/create-new-org.png b/sources/platform/8-collaboration/images/organizations/create-new-org.png similarity index 100% rename from sources/platform/collaboration/images/organizations/create-new-org.png rename to sources/platform/8-collaboration/images/organizations/create-new-org.png diff --git a/sources/platform/collaboration/images/organizations/integrations.png b/sources/platform/8-collaboration/images/organizations/integrations.png similarity index 100% rename from sources/platform/collaboration/images/organizations/integrations.png rename to sources/platform/8-collaboration/images/organizations/integrations.png diff --git a/sources/platform/collaboration/images/organizations/members.png b/sources/platform/8-collaboration/images/organizations/members.png similarity index 100% rename from sources/platform/collaboration/images/organizations/members.png rename to sources/platform/8-collaboration/images/organizations/members.png diff --git a/sources/platform/collaboration/images/organizations/my-organizations.png b/sources/platform/8-collaboration/images/organizations/my-organizations.png similarity index 100% rename from sources/platform/collaboration/images/organizations/my-organizations.png rename to sources/platform/8-collaboration/images/organizations/my-organizations.png diff --git a/sources/platform/collaboration/images/organizations/roles.png b/sources/platform/8-collaboration/images/organizations/roles.png similarity index 100% rename from sources/platform/collaboration/images/organizations/roles.png rename to sources/platform/8-collaboration/images/organizations/roles.png diff --git a/sources/platform/collaboration/images/organizations/switch-to-organization.png b/sources/platform/8-collaboration/images/organizations/switch-to-organization.png similarity index 100% rename from sources/platform/collaboration/images/organizations/switch-to-organization.png rename to sources/platform/8-collaboration/images/organizations/switch-to-organization.png diff --git a/sources/platform/collaboration/index.md b/sources/platform/8-collaboration/index.md similarity index 76% rename from sources/platform/collaboration/index.md rename to sources/platform/8-collaboration/index.md index 18aad02362..73a7c293e1 100644 --- a/sources/platform/collaboration/index.md +++ b/sources/platform/8-collaboration/index.md @@ -1,8 +1,7 @@ --- -title: Collaboration -description: Learn how to collaborate with other users and manage permissions for organizations or private resources such as Actors, Actor runs, and storages. -sidebar_position: 12 -category: platform +title: "Collaboration" +description: "Team management and access rights" +sidebar_position: 1 slug: /collaboration --- @@ -11,11 +10,11 @@ slug: /collaboration --- Apify was built from the ground up as a collaborative platform. Whether you’re publishing your Actor in Apify Store or sharing a dataset with a teammate, collaboration is deeply integrated into how Apify works. You can share your resources (like Actors, runs, or storages) with others, manage permissions, or invite collaborators to your organization. By default, each system resource you create is only available to you, the owner. However, you can grant access to other users, making it easy to collaborate effectively and securely. -While most resources can be shared by assigning permissions (see [Access Rights](./access_rights.md)), some resources can also be shared simply by using their unique links or IDs. There are two types of resources in terms of sharing: +While most resources can be shared by assigning permissions (see [Access Rights](/platform/collaboration/access-rights)), some resources can also be shared simply by using their unique links or IDs. There are two types of resources in terms of sharing: - _Resources that require explicit access by default:_ - - [Actors](../actors/running/index.md), [tasks](../actors/running/tasks.md) - - Can be shared only by inviting collaborators using [Access Rights](./access_rights.md)) or using [Organization Accounts](./organization_account/index.md) + - [Actors](/platform/using-actors), [tasks](///platform/using-actors/tasks) + - Can be shared only by inviting collaborators using [Access Rights](/platform/collaboration/access-rights)) or using [Organization Accounts](/platform/collaboration/organization-account) - _Resources supporting both explicit access and link sharing:_ - Actor runs, Actor builds and storage resources (datasets, key-value stores, request queues) - Can be shared by inviting collaborators or simply by sharing a unique direct link @@ -36,7 +35,7 @@ You can control access to your resources in four ways: Apify's organization account allows multiple engineers to collaborate on team projects with role-specific access permissions. - Publishing in Apify Store - Another way to share your Actor with other users is to publish it in Apify Store. When publishing your Actor, you can make it a Paid Actor and get paid by the users benefiting from your tool. For more information, read the publishing and monetization section. + Publishing in Apify Store + Another way to share your Actor with other users is to publish it in Apify Store. When publishing your Actor, you can make it a Paid Actor and get paid by the users benefiting from your tool. For more information, read the publishing and monetization section. diff --git a/sources/platform/collaboration/list_of_permissions.md b/sources/platform/8-collaboration/list_of_permissions.md similarity index 85% rename from sources/platform/collaboration/list_of_permissions.md rename to sources/platform/8-collaboration/list_of_permissions.md index abd7e1cceb..9e05f051dd 100644 --- a/sources/platform/collaboration/list_of_permissions.md +++ b/sources/platform/8-collaboration/list_of_permissions.md @@ -13,7 +13,7 @@ This document contains all the access options that can be granted to resources o ## Actors -To learn about Apify Actors, check out the [documentation](../actors/index.mdx). +To learn about Apify Actors, check out the [documentation](/platform/using-actors). ### Actor @@ -34,11 +34,11 @@ To learn about Apify Actors, check out the [documentation](../actors/index.mdx). | View runs | View a list of Actor task runs and their details. | | Manage access rights | Manage Actor task access rights. | -To learn about Actor tasks, see the [documentation](../actors/running/tasks.md). +To learn about Actor tasks, see the [documentation](///platform/using-actors/tasks). ## Storage -For more information about Storage, see its [documentation](../storage/index.md). +For more information about Storage, see its [documentation](/platform/core-concepts/storage). ### Dataset @@ -48,7 +48,7 @@ For more information about Storage, see its [documentation](../storage/index.md) | Write | Edit dataset settings, push data to it, and remove the dataset. | | Manage access rights | Manage dataset access rights. | -To learn about dataset storage, see its [documentation](../storage/dataset.md). +To learn about dataset storage, see its [documentation](/platform/core-concepts/storage/dataset). ### Key-value-store @@ -58,7 +58,7 @@ To learn about dataset storage, see its [documentation](../storage/dataset.md). | Write | Edit key-value store settings, add, update or remove its records, and delete the key-value store. | | Manage access rights | Manage key-value store access rights. | -To learn about key-value stores, see the [documentation](../storage/key_value_store.md). +To learn about key-value stores, see the [documentation](/platform/core-concepts/storage/key-value-store). ### Request queue @@ -68,7 +68,7 @@ To learn about key-value stores, see the [documentation](../storage/key_value_st | Write | Edit request queue settings, add, update, or remove its records, and delete the request queue. | | Manage access rights | Manage request queue access rights. | -To learn about request queue storage, see the [documentation](../storage/request_queue.md). +To learn about request queue storage, see the [documentation](/platform/core-concepts/storage/request-queue). ## Proxy @@ -76,11 +76,11 @@ To learn about request queue storage, see the [documentation](../storage/request |------------|---------------------------| | Proxy | Allow to use Apify Proxy. | -To learn about Apify Proxy, see its [documentation](../proxy/index.md). +To learn about Apify Proxy, see its [documentation](/platform/core-concepts/proxy). ## User permissions -Permissions that can be granted to members of organizations. To learn about the organization account, see its [documentation](./organization_account/index.md). +Permissions that can be granted to members of organizations. To learn about the organization account, see its [documentation](/platform/collaboration/organization-account). | Permission | Description | |---------------------|-----------------------------------------------------------------------| diff --git a/sources/platform/collaboration/organization_account/how_to_use.md b/sources/platform/8-collaboration/organization_account/how_to_use.md similarity index 92% rename from sources/platform/collaboration/organization_account/how_to_use.md rename to sources/platform/8-collaboration/organization_account/how_to_use.md index ee54ae198b..495796afef 100644 --- a/sources/platform/collaboration/organization_account/how_to_use.md +++ b/sources/platform/8-collaboration/organization_account/how_to_use.md @@ -24,7 +24,7 @@ You can switch into **Organization account** view using the account button in th In the menu, the account you are currently using is displayed at the top, with all the accounts you can switch to displayed below. When you need to get back to your personal account, you can just switch right back to it—no need to log in and out. -The resources you can access and account details you can edit will depend on your [permissions](../list_of_permissions.md) in the organization. +The resources you can access and account details you can edit will depend on your [permissions](/platform/collaboration/list-of-permissions) in the organization. > When switching between accounts, beware which account you start an Actor run in. If you accidentally start an organization's Actor run in your personal account, the run will be billed to your account (and vice versa). @@ -44,7 +44,7 @@ The organization, its Actors, and its integrations will keep running as they are While you cannot manage an organization account's settings and members via API, you can access its Actor and task runs, webhooks, schedules, and storages just as you would with any other account. -As a member of an organization, you are assigned an [API token](../../integrations/index.mdx) (under the **Integrations** tab) and proxy password (click the **Proxy** button in the left menu) for accessing the Apify platform via REST API. +As a member of an organization, you are assigned an [API token](/platform/integrations) (under the **Integrations** tab) and proxy password (click the **Proxy** button in the left menu) for accessing the Apify platform via REST API. ![Integration tokens](../images/organizations/integrations.png) diff --git a/sources/platform/collaboration/organization_account/index.md b/sources/platform/8-collaboration/organization_account/index.md similarity index 77% rename from sources/platform/collaboration/organization_account/index.md rename to sources/platform/8-collaboration/organization_account/index.md index 32842512ec..f915948dc4 100644 --- a/sources/platform/collaboration/organization_account/index.md +++ b/sources/platform/8-collaboration/organization_account/index.md @@ -9,14 +9,14 @@ slug: /collaboration/organization-account --- -Organization accounts allow groups to collaborate on projects. It enables you to manage your team members' [permissions](../list_of_permissions.md) and to centralize your billing without having to share the credentials of a single personal account. +Organization accounts allow groups to collaborate on projects. It enables you to manage your team members' [permissions](/platform/collaboration/list-of-permissions) and to centralize your billing without having to share the credentials of a single personal account. -You can [switch](./how_to_use.md) between your personal and organization accounts in just two clicks: in [Apify Console](https://console.apify.com), click the account button in the top-left corner, then select the organization. +You can [switch](/platform/collaboration/organization-account/how-to-use) between your personal and organization accounts in just two clicks: in [Apify Console](https://console.apify.com), click the account button in the top-left corner, then select the organization. You can set up an organization in two ways. * [Create a new organization](#create-a-new-organization). If you don't have integrations set up yet, or if they are easy to change, you can create a new organization, preserving your personal account. -* [Convert an existing account](#convert-an-existing-account) into an organization. If your Actors and [integrations](../../integrations/index.mdx) are set up in a personal account, it is probably best to convert that account into an organization. This will preserve all your integrations but means you will have a new personal account created for you. +* [Convert an existing account](#convert-an-existing-account) into an organization. If your Actors and [integrations](/platform/integrations) are set up in a personal account, it is probably best to convert that account into an organization. This will preserve all your integrations but means you will have a new personal account created for you. > Prefer video to reading? [See our video tutorial](https://www.youtube.com/watch?v=BIL6HqtnvKk) for organization accounts. @@ -38,7 +38,7 @@ You can create a new organization by clicking the **Create new organization** bu > > * **You will no longer be able to sign in to the converted user account.** > * **An organization cannot be converted back to a personal account.** -> * **During conversion, a new account (with the same login credentials) will be created for you. You can then use that account to [set up](./setup.md) the organization.** +> * **During conversion, a new account (with the same login credentials) will be created for you. You can then use that account to [set up](/platform/collaboration/organization-account/setup) the organization.** Before converting your personal account into an organization, make sure it has a **username**. @@ -52,7 +52,7 @@ Next, set the organization's name and click **Convert**. And that's it! Your personal account becomes the organization, and you will be logged out automatically. You can now log into your new personal account with the same credentials as you are currently logged in with. This applies to both **password** and **OAuth** methods. -For information on [adding members and assigning roles](./setup.md), see the Setup page. +For information on [adding members and assigning roles](/platform/collaboration/organization-account/setup), see the Setup page. ## Billing {#billing} diff --git a/sources/platform/collaboration/organization_account/setup.md b/sources/platform/8-collaboration/organization_account/setup.md similarity index 85% rename from sources/platform/collaboration/organization_account/setup.md rename to sources/platform/8-collaboration/organization_account/setup.md index ab93a7b29e..88c043cc08 100644 --- a/sources/platform/collaboration/organization_account/setup.md +++ b/sources/platform/8-collaboration/organization_account/setup.md @@ -41,10 +41,10 @@ To edit the permissions for each role, click on the **Configure permissions** bu > Each member can only have one role to avoid conflicting permissions. -You can configure individual permissions for each resource type such as Actors, Actor tasks or storage. Bear in mind that if a user has the **read** permission for [storages](../../storage/index.md), you cannot prevent them from accessing a particular storage (e.g. a certain [dataset](../../storage/index.md)) - they will have access to all of the organization's storages. +You can configure individual permissions for each resource type such as Actors, Actor tasks or storage. Bear in mind that if a user has the **read** permission for [storages](/platform/core-concepts/storage), you cannot prevent them from accessing a particular storage (e.g. a certain [dataset](/platform/core-concepts/storage)) - they will have access to all of the organization's storages. **Some permissions have dependencies**. For example, if someone has the **Actor run** permission, it is likely they will also need the **storage write** permission, so they can store the results from their Actor runs. ![Configure permissions](../images/organizations/configure-permissions.png) -[See the full list of permissions](../list_of_permissions.md) that can be granted to Apify resources. +[See the full list of permissions](/platform/collaboration/list-of-permissions) that can be granted to Apify resources. diff --git a/sources/platform/9-monitoring/_category_.yml b/sources/platform/9-monitoring/_category_.yml new file mode 100644 index 0000000000..7205da2906 --- /dev/null +++ b/sources/platform/9-monitoring/_category_.yml @@ -0,0 +1,2 @@ +label: 'Monitoring' +position: 9 diff --git a/sources/platform/monitoring/images/alerts.png b/sources/platform/9-monitoring/images/alerts.png similarity index 100% rename from sources/platform/monitoring/images/alerts.png rename to sources/platform/9-monitoring/images/alerts.png diff --git a/sources/platform/monitoring/images/daily-run-statuses.png b/sources/platform/9-monitoring/images/daily-run-statuses.png similarity index 100% rename from sources/platform/monitoring/images/daily-run-statuses.png rename to sources/platform/9-monitoring/images/daily-run-statuses.png diff --git a/sources/platform/monitoring/images/email-notification.png b/sources/platform/9-monitoring/images/email-notification.png similarity index 100% rename from sources/platform/monitoring/images/email-notification.png rename to sources/platform/9-monitoring/images/email-notification.png diff --git a/sources/platform/monitoring/images/in-app-notification.png b/sources/platform/9-monitoring/images/in-app-notification.png similarity index 100% rename from sources/platform/monitoring/images/in-app-notification.png rename to sources/platform/9-monitoring/images/in-app-notification.png diff --git a/sources/platform/monitoring/images/metric-options.png b/sources/platform/9-monitoring/images/metric-options.png similarity index 100% rename from sources/platform/monitoring/images/metric-options.png rename to sources/platform/9-monitoring/images/metric-options.png diff --git a/sources/platform/monitoring/images/monitoring.png b/sources/platform/9-monitoring/images/monitoring.png similarity index 100% rename from sources/platform/monitoring/images/monitoring.png rename to sources/platform/9-monitoring/images/monitoring.png diff --git a/sources/platform/monitoring/images/notifications.png b/sources/platform/9-monitoring/images/notifications.png similarity index 100% rename from sources/platform/monitoring/images/notifications.png rename to sources/platform/9-monitoring/images/notifications.png diff --git a/sources/platform/monitoring/images/run-statistics-chart.png b/sources/platform/9-monitoring/images/run-statistics-chart.png similarity index 100% rename from sources/platform/monitoring/images/run-statistics-chart.png rename to sources/platform/9-monitoring/images/run-statistics-chart.png diff --git a/sources/platform/monitoring/index.md b/sources/platform/9-monitoring/index.md similarity index 87% rename from sources/platform/monitoring/index.md rename to sources/platform/9-monitoring/index.md index 4cb75ef4f0..369f6a6526 100644 --- a/sources/platform/monitoring/index.md +++ b/sources/platform/9-monitoring/index.md @@ -1,8 +1,7 @@ --- title: Monitoring -description: Learn how to continuously make sure that your Actors and tasks perform as expected and retrieve correct results. Receive alerts when your jobs or their metrics are not as you expect. -sidebar_position: 12 -category: guides +description: "Learn how to continuously make sure that your Actors and tasks perform as expected and retrieve correct results. Receive alerts when your jobs or their metrics are not as you expect." +sidebar_position: 1 slug: /monitoring --- @@ -49,7 +48,7 @@ When you set up an alert, you have four choices for how you want the metrics to 3. **Alert, when run status is one of following** - This type of alert is checked only after the run finishes. It makes possible to track the status of your finished runs and send an alert if the run finishes in a state you do not expect. If your Actor runs very often and suddenly starts failing, you will receive a single alert after the first failed run in 1 minute, and then aggregated alert every 15 minutes. -4. **Alert for dataset field statistics** - If you have a [dataset schema](../actors/development/actor_definition/dataset_schema/validation.md) set up, then you can use the field statistics to set up an alert. You can use field statistics for example to track if some field is filled in all records, if some numeric value is too low/high (for example when tracking the price of a product over multiple sources), if the number of items in an array is too low/high (for example alert on Instagram Actor if post has a lot of comments) and many other tasks like these. +4. **Alert for dataset field statistics** - If you have a [dataset schema](//platform/building-actors/actor-definition/dataset-schema/validation) set up, then you can use the field statistics to set up an alert. You can use field statistics for example to track if some field is filled in all records, if some numeric value is too low/high (for example when tracking the price of a product over multiple sources), if the number of items in an array is too low/high (for example alert on Instagram Actor if post has a lot of comments) and many other tasks like these. :::important @@ -108,7 +107,7 @@ These are just a few examples of what you can monitor. It's always recommended t ### Alternative solutions -For more complex monitoring, you can use the [monitoring suite](https://apify.com/apify/monitoring), which is a collection of [Apify Actors](../actors/index.mdx) that allows you to automate the monitoring of jobs you have running on the [Apify platform](https://apify.com). The monitoring suite offers some features that are not **currently** available in Apify Console, such as: +For more complex monitoring, you can use the [monitoring suite](https://apify.com/apify/monitoring), which is a collection of Apify Actors that allows you to automate the monitoring of jobs you have running on the [Apify platform](https://apify.com). The monitoring suite offers some features that are not **currently** available in Apify Console, such as: 1. Schema validation of the output 2. Duplicate checks in the output diff --git a/sources/platform/actors/development/index.md b/sources/platform/actors/development/index.md deleted file mode 100644 index 3a97539d70..0000000000 --- a/sources/platform/actors/development/index.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -title: Actor development -desc: Read about the technical part of building Apify Actors. Learn to define Actor inputs, build new versions, persist Actor state, and choose base Docker images. -sidebar_label: Development -sidebar_position: 7.4 -slug: /actors/development ---- - -**Read about the technical part of building Apify Actors. Learn to define Actor inputs, build new versions, persist Actor state, and choose base Docker images.** - ---- - -This section will guide you through the whole story of [Actor](../index.mdx) development. - -You can follow chapters sequentially from [Quick start](./quick_start/index.mdx), where you learn how to create your first Actor in just a few minutes, through the more technical sections describing the whole Actor model, up to the [Performance](/sources/platform/actors/development/performance.md) section, where you learn how to fine-tune your Actor to get the most out of the Apify platform. - -import Card from "@site/src/components/Card"; -import CardGrid from "@site/src/components/CardGrid"; - - - - - - - - - - ---- - -If your Actor allows for integration with external services, or performs some post-processing of datasets generated by other Actors, check out the section [integration-ready Actors](../../integrations/actors/integration_ready_actors.md). - -After your development, you can jump to the section [Publishing and monetization](../publishing/index.mdx) to learn about how to publish your Actor in [Apify Store](https://apify.com/store) and monetize it by renting it out to users of the platform. diff --git a/sources/platform/actors/images/apify-actor-drawing.png b/sources/platform/actors/images/apify-actor-drawing.png deleted file mode 100644 index 1e57b23cef..0000000000 Binary files a/sources/platform/actors/images/apify-actor-drawing.png and /dev/null differ diff --git a/sources/platform/actors/images/run-console.png b/sources/platform/actors/images/run-console.png deleted file mode 100644 index 055e1fe8aa..0000000000 Binary files a/sources/platform/actors/images/run-console.png and /dev/null differ diff --git a/sources/platform/actors/index.mdx b/sources/platform/actors/index.mdx deleted file mode 100644 index 6586f43d3a..0000000000 --- a/sources/platform/actors/index.mdx +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Actors -description: Learn how to develop, run and share serverless cloud programs. Create your own web scraping and automation tools and publish them on the Apify platform. -sidebar_position: 7 -category: platform -slug: /actors ---- - -**Learn how to develop, run and share serverless cloud programs. Create your own web scraping and automation tools and publish them on the Apify platform.** - -import Card from "@site/src/components/Card"; -import CardGrid from "@site/src/components/CardGrid"; - ---- - - - - - - - -## Actors overview - -Actors are serverless cloud programs that can perform anything from a simple action, like filling out a web form, to a complex operation, like crawling an entire website or removing duplicates from a large dataset. Because Actors can persist their state and be restarted, their runs can be as short or as long as necessary, from seconds to hours, or even indefinitely. - -Basically, Actors are programs packaged as Docker images, which accept a well-defined JSON input, perform an action, and optionally produce a well-defined JSON output. - -:::info Additional context - -For more context, read the [Actor whitepaper](https://whitepaper.actor/). - -::: - -## Actor components - -Actors consist of these elements: - -- _Dockerfile_ which specifies where the Actor's source code is, how to build it, and run it. -- _Documentation_ in a form of a README.md file. -- _Input and output schemas_ that describe what input the Actor requires, and what results it produces. -- Access to an out-of-the-box _storage system_ for Actor data, results, and files. -- _Metadata_ such as the Actor name, description, author, and version. - -The documentation and input/output schemas help people understand what the Actor does, enter required inputs in the user interface or API, and integrate results into other workflows. Actors can call and interact with each other to build more complex systems from simple ones. - -![Apify Actor diagram](./images/apify-actor-drawing.png) - -## Build Actors - -Build Actors to automate tasks, scrape data, or create custom workflows. The Apify platform gives you everything you need to develop, test, and deploy your code. - -Ready to start? Check out the [Actor development documentation](/platform/actors/development). - -## Running Actors - -You can run Actors manually in [Apify Console](https://console.apify.com/actors), using the [API](/api), [CLI](/cli), or [scheduler](../schedules.md). You can easily [integrate Actors](../integrations/index.mdx) with other apps, [share](../collaboration/access_rights.md) them with other people, [publish](./publishing/index.mdx) them in [Apify Store](https://apify.com/store), and even [monetize](./publishing/monetize/index.mdx). - -:::tip Try Actors - -To get a better idea of what Apify Actors are, visit [Apify Store](https://apify.com/store) and try out some of them! - -![Apify Store](./running/images/store-google-maps-scraper.png) - -::: - - -## Public and private Actors - -Actors can be [public](./actors/running/actors-in-store) or private. Private Actors are yours to use and keep; no one will see them if you don't want them to. Public Actors are [available to everyone](./running/store.md) in [Apify Store](https://apify.com/store). You can make them free to use, or you can [charge for them](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/). diff --git a/sources/platform/homepage_content.json b/sources/platform/homepage_content.json index 2c5fc61f78..be416fc8f6 100644 --- a/sources/platform/homepage_content.json +++ b/sources/platform/homepage_content.json @@ -1,23 +1,18 @@ [ - { - "title": "Actors", - "description": "Learn how to develop, run and share own web scraping and automation tools.", - "to": "/platform/actors" - }, { "title": "Storage", "description": "Store anything from images and key-value pairs to structured output data.", - "to": "/platform/storage" + "to": "/platform/core-concepts/storage" }, { "title": "Proxy", "description": "Learn to anonymously and reliably access websites in scraping/automation jobs.", - "to": "/platform/proxy" + "to": "/platform/core-concepts/proxy" }, { "title": "Schedules", "description": "Learn how to automatically start your Actor and task runs.", - "to": "/platform/schedules" + "to": "/platform/using-actors/schedules" }, { "title": "Integrations", diff --git a/sources/platform/index.mdx b/sources/platform/index.mdx index 50204a168f..5411e67c48 100644 --- a/sources/platform/index.mdx +++ b/sources/platform/index.mdx @@ -22,12 +22,12 @@ import homepageContent from "./homepage_content.json"; [Apify Proxy](https://apify.com/proxy) allows you to change your IP address when web scraping to reduce the chance of being [blocked](/academy/anti-scraping/techniques) because of your geographical location. - -You can use proxies in your [Actors](../actors/index.mdx) or any other application that supports HTTP proxies. Apify Proxy monitors the health of your IP pool and intelligently rotates addresses to prevent IP address-based blocking. - -You can view your proxy settings and password on the [Proxy](https://console.apify.com/proxy) page in Apify Console. For pricing information, visit [apify.com/pricing](https://apify.com/pricing). - - -## Quickstart - -Usage of Apify Proxy means just a couple of lines of code, thanks to our [SDKs](/sdk): - - - - -```javascript -import { Actor } from 'apify'; -import { PuppeteerCrawler } from 'crawlee'; - -await Actor.init(); - -const proxyConfiguration = await Actor.createProxyConfiguration(); - -const crawler = new PuppeteerCrawler({ - proxyConfiguration, - async requestHandler({ page }) { - console.log(await page.content()); - }, -}); - -await crawler.run(['https://proxy.apify.com/?format=json']); - -await Actor.exit(); -``` - - - - -```python -import requests, asyncio -from apify import Actor - -async def main(): - async with Actor: - proxy_configuration = await Actor.create_proxy_configuration() - proxy_url = await proxy_configuration.new_url() - - proxies = { - 'http': proxy_url, - 'https': proxy_url, - } - - response = requests.get('https://api.apify.com/v2/browser-info', proxies=proxies) - print(response.text) - -if __name__ == '__main__': - asyncio.run(main()) -``` - - - - -## Proxy types - -Several types of proxy servers exist, each offering distinct advantages, disadvantages, and varying pricing structures. You can use them to access websites from various geographies and with different levels of anonymity. - - - - - - - diff --git a/sources/platform/security.md b/sources/platform/security.md index 6ac92f47f0..a26d3f8345 100644 --- a/sources/platform/security.md +++ b/sources/platform/security.md @@ -1,8 +1,7 @@ --- -title: Security -description: Learn more about Apify's security practices and data protection measures that are used to protect your Actors, their data, and the Apify platform in general. -sidebar_position: 15 -category: platform +title: "Security" +description: "Learn more about Apify's security practices and data protection measures that are used to protect your Actors, their data, and the Apify platform in general." +sidebar_position: 10 slug: /security --- @@ -111,4 +110,4 @@ Please adhere strictly to the following rules. Failure to do so may result in le ## Securing your data -The Apify platform provides you with multiple ways to secure your data, including [encrypted environment variables](./actors/development/programming_interface/environment_variables.md) for storing your configuration secrets and [encrypted input](./actors/development/actor_definition/input_schema/secret_input.md) for securing the input parameters of your Actors. +The Apify platform provides you with multiple ways to secure your data, including [encrypted environment variables](//platform/building-actors/programming-interface/environment-variables) for storing your configuration secrets and [encrypted input](//platform/building-actors/actor-definition/input-schema/secret-input) for securing the input parameters of your Actors. diff --git a/sources/platform/storage/index.md b/sources/platform/storage/index.md deleted file mode 100644 index 5682d4a8f7..0000000000 --- a/sources/platform/storage/index.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Storage -description: Store anything from images and key-value pairs to structured output data. Learn how to access and manage your stored data from the Apify platform or via API. -sidebar_position: 9 -category: platform -slug: /storage ---- - -import Card from "@site/src/components/Card"; -import CardGrid from "@site/src/components/CardGrid"; -import StoragePricingCalculator from "@site/src/components/StoragePricingCalculator"; - -**Store anything from images and key-value pairs to structured output data. Learn how to access and manage your stored data on the Apify Console or via the API.** - ---- - -The Apify platform provides three types of storage accessible both within our [Apify Console](https://console.apify.com/storage) and externally through our [REST API](/api/v2) [Apify API Clients](/api) or [SDKs](/sdk). - - - - - - - -