Skip to content

Commit fb8514b

Browse files
vdusekvdusek
authored
docs: Move code samples to files and other updates (#340)
### Description - Relocated all code samples to dedicated files for being able to perform static analysis. - Updated home page example and moved it to a dedicated file. - Updated `pyproject.toml` configurations to include only directories with source files (`website/` is added there, same as in Crawlee and SDK). - Revised most of the documentation content. - All code examples, where it makes sense, are both in sync & async forms. ### Issues - Closes: #337 ### Testing - All code samples pass checks with Ruff and Mypy. ### Checklist - [x] CI passed
1 parent 5ae15ec commit fb8514b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+1398
-515
lines changed

‎Makefile‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,6 @@
22
integration-tests format check-async-docstrings check-code fix-async-docstrings \
33
build-api-reference build-docs run-docs
44

5-
DIRS_WITH_CODE = src tests scripts
6-
75
# This is default for local testing, but GitHub workflows override it to a higher value in CI
86
INTEGRATION_TESTS_CONCURRENCY = 1
97

@@ -23,11 +21,11 @@ publish-to-pypi:
2321
poetry publish --no-interaction -vv
2422

2523
lint:
26-
poetry run ruff format --check $(DIRS_WITH_CODE)
27-
poetry run ruff check $(DIRS_WITH_CODE)
24+
poetry run ruff format --check
25+
poetry run ruff check
2826

2927
type-check:
30-
poetry run mypy $(DIRS_WITH_CODE)
28+
poetry run mypy
3129

3230
unit-tests:
3331
poetry run pytest --numprocesses=auto --verbose --cov=src/apify_client tests/unit
@@ -39,8 +37,8 @@ integration-tests:
3937
poetry run pytest --numprocesses=$(INTEGRATION_TESTS_CONCURRENCY) tests/integration
4038

4139
format:
42-
poetry run ruff check --fix $(DIRS_WITH_CODE)
43-
poetry run ruff format $(DIRS_WITH_CODE)
40+
poetry run ruff check --fix
41+
poetry run ruff format
4442

4543
check-async-docstrings:
4644
poetry run python scripts/check_async_docstrings.py

‎docs/01_overview/01_introduction.mdx‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
id: introduction
3+
title: Introduction
4+
---
5+
6+
import Tabs from '@theme/Tabs';
7+
import TabItem from '@theme/TabItem';
8+
import CodeBlock from '@theme/CodeBlock';
9+
10+
import UsageAsyncExample from '!!raw-loader!./code/01_usage_async.py';
11+
import UsageSyncExample from '!!raw-loader!./code/01_usage_sync.py';
12+
13+
The [Apify client for Python](https://github.com/apify/apify-client-python) is the official library to access the [Apify REST API](https://docs.apify.com/api/v2) from your Python applications. It provides useful features like automatic retries and convenience functions that improve the experience of using the Apify API. All requests and responses (including errors) are encoded in JSON format with UTF-8 encoding. The client provides both synchronous and asynchronous interfaces.
14+
15+
<Tabs>
16+
<TabItem value="AsyncExample" label="Async client" default>
17+
<CodeBlock className="language-python">
18+
{UsageAsyncExample}
19+
</CodeBlock>
20+
</TabItem>
21+
<TabItem value="SyncExample" label="Sync client">
22+
<CodeBlock className="language-python">
23+
{UsageSyncExample}
24+
</CodeBlock>
25+
</TabItem>
26+
</Tabs>

‎docs/01_overview/02_setting_up.mdx‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
id: setting-up
3+
title: Setting up
4+
---
5+
6+
import Tabs from '@theme/Tabs';
7+
import TabItem from '@theme/TabItem';
8+
import CodeBlock from '@theme/CodeBlock';
9+
10+
import AsyncExample from '!!raw-loader!./code/02_auth_async.py';
11+
import SyncExample from '!!raw-loader!./code/02_auth_sync.py';
12+
13+
This guide will help you get started with [Apify client for Python](https://github.com/apify/apify-client-python) by setting it up on your computer. Follow the steps below to ensure a smooth installation process.
14+
15+
## Prerequisites
16+
17+
Before installing `apify-client` itself, make sure that your system meets the following requirements:
18+
19+
- **Python 3.9 or higher**: `apify-client` requires Python 3.9 or a newer version. You can download Python from the [official website](https://www.python.org/downloads/).
20+
- **Python package manager**: While this guide uses Pip (the most common package manager), you can also use any package manager you want. You can download Pip from the [official website](https://pip.pypa.io/en/stable/installation/).
21+
22+
### Verifying prerequisites
23+
24+
To check if Python and the Pip package manager are installed, run the following commands:
25+
26+
```sh
27+
python --version
28+
```
29+
30+
```sh
31+
pip --version
32+
```
33+
34+
If these commands return the respective versions, you're ready to continue.
35+
36+
## Installation
37+
38+
Apify client for Python is available as the [`apify-client`](https://pypi.org/project/crawlee/) package on PyPI. To install it, run:
39+
40+
```sh
41+
pip install apify-client
42+
```
43+
44+
After installation, verify that `apify-client` is installed correctly by checking its version:
45+
46+
```sh
47+
python -c 'import apify_client; print(apify_client.__version__)'
48+
```
49+
50+
## Authentication and initialization
51+
52+
To use the client, you need an [API token](https://docs.apify.com/platform/integrations/api#api-token). You can find your token under [Integrations](https://console.apify.com/account/integrations) tab in Apify Console. Copy the token and initialize the client by providing the token (`MY-APIFY-TOKEN`) as a parameter to the `ApifyClient` constructor.
53+
54+
<Tabs>
55+
<TabItem value="AsyncExample" label="Async client" default>
56+
<CodeBlock className="language-python">
57+
{AsyncExample}
58+
</CodeBlock>
59+
</TabItem>
60+
<TabItem value="SyncExample" label="Sync client">
61+
<CodeBlock className="language-python">
62+
{SyncExample}
63+
</CodeBlock>
64+
</TabItem>
65+
</Tabs>
66+
67+
:::warning Secure access
68+
69+
The API token is used to authorize your requests to the Apify API. You can be charged for the usage of the underlying services, so do not share your API token with untrusted parties or expose it on the client side of your applications.
70+
71+
:::

‎docs/01_overview/03_getting_started.mdx‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
id: getting-started
3+
title: Getting started
4+
---
5+
6+
import Tabs from '@theme/Tabs';
7+
import TabItem from '@theme/TabItem';
8+
import CodeBlock from '@theme/CodeBlock';
9+
10+
import UsageAsyncExample from '!!raw-loader!./code/01_usage_async.py';
11+
import UsageSyncExample from '!!raw-loader!./code/01_usage_sync.py';
12+
import InputAsyncExample from '!!raw-loader!./code/03_input_async.py';
13+
import InputSyncExample from '!!raw-loader!./code/03_input_sync.py';
14+
import DatasetAsyncExample from '!!raw-loader!./code/03_dataset_async.py';
15+
import DatasetSyncExample from '!!raw-loader!./code/03_dataset_sync.py';
16+
17+
This guide will walk you through how to use the [Apify Client for Python](https://github.com/apify/apify-client-python) to run [Actors](https://apify.com/actors) on the [Apify platform](https://docs.apify.com/platform), provide input to them, and retrieve results from their datasets. You'll learn the basics of running serverless programs (we're calling them Actors) and managing their output efficiently.
18+
19+
## Running your first Actor
20+
21+
To start an Actor, you need its ID (e.g., `john-doe/my-cool-actor`) and an API token. The Actor's ID is a combination of the username and the Actor owner's username. Use the [`ActorClient`](/reference/class/ActorClient) to run the Actor and wait for it to complete. You can run both your own Actors and [Actors from Apify store](https://docs.apify.com/platform/actors/running/actors-in-store).
22+
23+
<Tabs>
24+
<TabItem value="AsyncExample" label="Async client" default>
25+
<CodeBlock className="language-python">
26+
{UsageAsyncExample}
27+
</CodeBlock>
28+
</TabItem>
29+
<TabItem value="SyncExample" label="Sync client">
30+
<CodeBlock className="language-python">
31+
{UsageSyncExample}
32+
</CodeBlock>
33+
</TabItem>
34+
</Tabs>
35+
36+
## Providing input to Actor
37+
38+
Actors often require input, such as URLs to scrape, search terms, or other configuration data. You can pass input as a JSON object when starting the Actor using the [`ActorClient.call`](/reference/class/ActorClient#call) method. Actors respect the input schema defined in the Actor's [input schema](https://docs.apify.com/platform/actors/development/actor-definition/input-schema).
39+
40+
<Tabs>
41+
<TabItem value="AsyncExample" label="Async client" default>
42+
<CodeBlock className="language-python">
43+
{InputAsyncExample}
44+
</CodeBlock>
45+
</TabItem>
46+
<TabItem value="SyncExample" label="Sync client">
47+
<CodeBlock className="language-python">
48+
{InputSyncExample}
49+
</CodeBlock>
50+
</TabItem>
51+
</Tabs>
52+
53+
## Getting results from the dataset
54+
55+
To get the results from the dataset, you can use the [`DatasetClient`](/reference/class/DatasetClient) ([`ApifyClient.dataset`](/reference/class/ApifyClient#dataset) ) and [`DatasetClient.list_items`](/reference/class/DatasetClient#list_items) method. You need to pass the dataset ID to define which dataset you want to access. You can get the dataset ID from the Actor's run dictionary (represented by `defaultDatasetId`).
56+
57+
<Tabs>
58+
<TabItem value="AsyncExample" label="Async client" default>
59+
<CodeBlock className="language-python">
60+
{InputAsyncExample}
61+
</CodeBlock>
62+
</TabItem>
63+
<TabItem value="SyncExample" label="Sync client">
64+
<CodeBlock className="language-python">
65+
{InputSyncExample}
66+
</CodeBlock>
67+
</TabItem>
68+
</Tabs>
69+
70+
:::note Dataset access
71+
72+
Running an Actor might take time, depending on the Actor's complexity and the amount of data it processes. If you want only to get data and have an immediate response you should access the existing dataset of the finished [Actor run](https://docs.apify.com/platform/actors/running/runs-and-builds#runs).
73+
74+
:::

‎docs/01_overview/code/01_usage_async.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
from apify_client import ApifyClientAsync
2+
3+
# You can find your API token at https://console.apify.com/settings/integrations.
4+
TOKEN = 'MY-APIFY-TOKEN'
5+
6+
7+
async def main() -> None:
8+
apify_client = ApifyClientAsync(TOKEN)
9+
10+
# Start an Actor and wait for it to finish.
11+
actor_client = apify_client.actor('john-doe/my-cool-actor')
12+
call_result = await actor_client.call()
13+
14+
if call_result is None:
15+
print('Actor run failed.')
16+
return
17+
18+
# Fetch results from the Actor run's default dataset.
19+
dataset_client = apify_client.dataset(call_result['defaultDatasetId'])
20+
list_items_result = await dataset_client.list_items()
21+
print(f'Dataset: {list_items_result}')

‎docs/01_overview/code/01_usage_sync.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
from apify_client import ApifyClient
2+
3+
# You can find your API token at https://console.apify.com/settings/integrations.
4+
TOKEN = 'MY-APIFY-TOKEN'
5+
6+
7+
def main() -> None:
8+
apify_client = ApifyClient(TOKEN)
9+
10+
# Start an Actor and wait for it to finish.
11+
actor_client = apify_client.actor('john-doe/my-cool-actor')
12+
call_result = actor_client.call()
13+
14+
if call_result is None:
15+
print('Actor run failed.')
16+
return
17+
18+
# Fetch results from the Actor run's default dataset.
19+
dataset_client = apify_client.dataset(call_result['defaultDatasetId'])
20+
list_items_result = dataset_client.list_items()
21+
print(f'Dataset: {list_items_result}')

‎docs/01_overview/code/02_auth_async.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
from apify_client import ApifyClientAsync
2+
3+
TOKEN = 'MY-APIFY-TOKEN'
4+
5+
6+
async def main() -> None:
7+
# Client initialization with the API token.
8+
apify_client = ApifyClientAsync(TOKEN)

‎docs/01_overview/code/02_auth_sync.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
from apify_client import ApifyClient
2+
3+
TOKEN = 'MY-APIFY-TOKEN'
4+
5+
6+
def main() -> None:
7+
# Client initialization with the API token.
8+
apify_client = ApifyClient(TOKEN)

‎docs/01_overview/code/03_dataset_async.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
from apify_client import ApifyClientAsync
2+
3+
TOKEN = 'MY-APIFY-TOKEN'
4+
5+
6+
async def main() -> None:
7+
apify_client = ApifyClientAsync(TOKEN)
8+
dataset_client = apify_client.dataset('dataset-id')
9+
10+
# Lists items from the Actor's dataset.
11+
dataset_items = (await dataset_client.list_items()).items

‎docs/01_overview/code/03_dataset_sync.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
from apify_client import ApifyClient
2+
3+
TOKEN = 'MY-APIFY-TOKEN'
4+
5+
6+
def main() -> None:
7+
apify_client = ApifyClient(TOKEN)
8+
dataset_client = apify_client.dataset('dataset-id')
9+
10+
# Lists items from the Actor's dataset.
11+
dataset_items = dataset_client.list_items().items

0 commit comments

Comments
 (0)