Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
8369ff9
Test with crawlee branch `storage-clients-and-configurations`
Pijukatel Aug 29, 2025
d9137aa
Add debug
Pijukatel Aug 29, 2025
cf1ee6f
Update config handling
Pijukatel Sep 1, 2025
d6b85ac
Add many configuration based tests
Pijukatel Sep 1, 2025
ea8e085
Add storage tests
Pijukatel Sep 1, 2025
9c3e7b1
Do Pydantic workaround
Pijukatel Sep 1, 2025
a2825bf
Wip, TODO: Solve patching of service_locator from Crawlee
Pijukatel Sep 2, 2025
0b96454
Update lock
Pijukatel Sep 2, 2025
432c79c
Remove any monkey patching from Configuration
Pijukatel Sep 10, 2025
a4a046e
Move all relevant initialization for Actor from __init__ to init to e…
Pijukatel Sep 10, 2025
2a52cdc
Update inits
Pijukatel Sep 10, 2025
841d89a
Update tests
Pijukatel Sep 10, 2025
8bd59fd
Fix failing tests
Pijukatel Sep 10, 2025
c4b5d48
Remove leftover edits
Pijukatel Sep 11, 2025
19ea5c7
Update init regarding the implicit config finalization
Pijukatel Sep 11, 2025
54a3523
Finalize tests
Pijukatel Sep 11, 2025
c89fd73
Merge remote-tracking branch 'origin/master' into test-new-storage-se…
Pijukatel Sep 11, 2025
b4efbff
Properly set implicit ApifyFileSystemStorageClient
Pijukatel Sep 11, 2025
6a6ab98
Update test
Pijukatel Sep 12, 2025
f1ce0d1
Review feedback
Pijukatel Sep 12, 2025
8347eb6
Merge remote-tracking branch 'origin/master' into test-new-storage-se…
Pijukatel Sep 12, 2025
b7101a4
Master related update
Pijukatel Sep 12, 2025
19b79f1
Add upgrading guide
Pijukatel Sep 12, 2025
b256876
Add migration test
Pijukatel Sep 12, 2025
6fbb5f4
Ensure proper storoage client init when is_at_home to avoid unnecesar…
Pijukatel Sep 12, 2025
5bf51f7
Add warning for usage of FileSystemStorageClient in Actor context
Pijukatel Sep 12, 2025
14c5395
Add special caching for ApifyClient
Pijukatel Sep 15, 2025
f7c9a58
Remove line that is no longer necessary
Pijukatel Sep 15, 2025
4450bf8
Update lock
Pijukatel Sep 15, 2025
f28fcd7
Merge remote-tracking branch 'origin/master' into test-new-storage-se…
Pijukatel Sep 16, 2025
c2c8ca5
Update NDU creation logic based on updated Crawlee
Pijukatel Sep 17, 2025
7911c48
Update tests
Pijukatel Sep 17, 2025
e68bdef
Update lock
Pijukatel Sep 17, 2025
04e74bc
Review comments
Pijukatel Sep 17, 2025
1cb295f
Do not attempt to deal with limited retention for alias storages locally
Pijukatel Sep 17, 2025
4b5946f
Add more docstrings
Pijukatel Sep 17, 2025
70890e7
Review call changes
Pijukatel Sep 17, 2025
2d61f1e
Review call changes 2
Pijukatel Sep 17, 2025
3813ea3
Add docs and compute_short_hash for additional cache key
Pijukatel Sep 18, 2025
5608b4d
Remove Actor.config
Pijukatel Sep 18, 2025
4b6c414
crawler actor reboot test
Pijukatel Sep 18, 2025
79b0ff7
Move test_apify_storages
Pijukatel Sep 18, 2025
b424c9b
Update test_configuration.py
Pijukatel Sep 18, 2025
cae107e
Update typing
Pijukatel Sep 18, 2025
177dbb2
Fix naming in failing test
Pijukatel Sep 18, 2025
698c089
Add warning to potential missuse of Configuration
Pijukatel Sep 18, 2025
3b7634a
Update caplog test
Pijukatel Sep 18, 2025
0de39ad
Revert lock changes
Pijukatel Sep 18, 2025
b19ea8c
Review comments
Pijukatel Sep 18, 2025
85bf04d
Remove unused self._charging_manager
Pijukatel Sep 18, 2025
47f84eb
Apply suggestions from code review
Pijukatel Sep 18, 2025
7d1fc84
Review comments
Pijukatel Sep 18, 2025
35ad0dd
Remove difference in storage clients
Pijukatel Sep 18, 2025
28ebd8d
inline finalize
vdusek Sep 18, 2025
f884356
Order methods in AliasResolver
Pijukatel Sep 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/02_concepts/09_running_webserver.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ The URL is available in the following places:

- In Apify Console, on the Actor run details page as the **Container URL** field.
- In the API as the `container_url` property of the [Run object](https://docs.apify.com/api/v2#/reference/actors/run-object/get-run).
- In the Actor as the `Actor.config.container_url` property.
- In the Actor as the `Actor.configuration.container_url` property.

The web server running inside the container must listen at the port defined by the `Actor.config.container_port` property. When running Actors locally, the port defaults to `4321`, so the web server will be accessible at `http://localhost:4321`.
The web server running inside the container must listen at the port defined by the `Actor.configuration.container_port` property. When running Actors locally, the port defaults to `4321`, so the web server will be accessible at `http://localhost:4321`.

## Example

Expand Down
2 changes: 1 addition & 1 deletion docs/02_concepts/code/07_webhook_preventing.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ async def main() -> None:
webhook = Webhook(
event_types=['ACTOR.RUN.FAILED'],
request_url='https://example.com/run-failed',
idempotency_key=Actor.config.actor_run_id,
idempotency_key=Actor.configuration.actor_run_id,
)

# Add the webhook to the Actor.
Expand Down
4 changes: 2 additions & 2 deletions docs/02_concepts/code/09_webserver.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ def run_server() -> None:
# and save a reference to the server.
global http_server
with ThreadingHTTPServer(
('', Actor.config.web_server_port), RequestHandler
('', Actor.configuration.web_server_port), RequestHandler
) as server:
Actor.log.info(f'Server running on {Actor.config.web_server_port}')
Actor.log.info(f'Server running on {Actor.configuration.web_server_port}')
http_server = server
server.serve_forever()

Expand Down
2 changes: 1 addition & 1 deletion docs/02_concepts/code/conditional_actor_charge.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ async def main() -> None:
if Actor.get_charging_manager().get_pricing_info().is_pay_per_event:
# highlight-end
await Actor.push_data({'hello': 'world'}, 'dataset-item')
elif charged_items < (Actor.config.max_paid_dataset_items or 0):
elif charged_items < (Actor.configuration.max_paid_dataset_items or 0):
await Actor.push_data({'hello': 'world'})
charged_items += 1
2 changes: 1 addition & 1 deletion docs/03_guides/code/03_playwright.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ async def main() -> None:
async with async_playwright() as playwright:
# Configure the browser to launch in headless mode as per Actor configuration.
browser = await playwright.chromium.launch(
headless=Actor.config.headless,
headless=Actor.configuration.headless,
args=['--disable-gpu'],
)
context = await browser.new_context()
Expand Down
2 changes: 1 addition & 1 deletion docs/03_guides/code/04_selenium.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ async def main() -> None:
Actor.log.info('Launching Chrome WebDriver...')
chrome_options = ChromeOptions()

if Actor.config.headless:
if Actor.configuration.headless:
chrome_options.add_argument('--headless')

chrome_options.add_argument('--no-sandbox')
Expand Down
36 changes: 36 additions & 0 deletions docs/04_upgrading/upgrading_to_v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,42 @@ This page summarizes the breaking changes between Apify Python SDK v2.x and v3.0

Support for Python 3.9 has been dropped. The Apify Python SDK v3.x now requires Python 3.10 or later. Make sure your environment is running a compatible version before upgrading.

## Actor initialization and ServiceLocator changes

`Actor` initialization and global `service_locator` services setup is more strict and predictable.
- Services in `Actor` can't be changed after calling `Actor.init`, entering the `async with Actor` context manager or after requesting them from the `Actor`.
- Services in `Actor` can be different from services in Crawler.


**Now (v3.0):**

```python
from crawlee.crawlers import BasicCrawler
from crawlee.storage_clients import MemoryStorageClient
from crawlee.configuration import Configuration
from crawlee.events import LocalEventManager
from apify import Actor

async def main():

async with Actor():
# This crawler will use same services as Actor and global service_locator
crawler_1 = BasicCrawler()

# This crawler will use custom services
custom_configuration = Configuration()
custom_event_manager = LocalEventManager.from_config(custom_configuration)
custom_storage_client = MemoryStorageClient()
crawler_2 = BasicCrawler(
configuration=custom_configuration,
event_manager=custom_event_manager,
storage_client=custom_storage_client,
)
```

## Removed Actor.config property
- `Actor.config` property has been removed. Use `Actor.configuration` instead.

## Storages

<!-- TODO -->
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ keywords = [
dependencies = [
"apify-client>=2.0.0,<3.0.0",
"apify-shared>=2.0.0,<3.0.0",
"crawlee==0.6.13b37",
"crawlee==0.6.13b42",
"cachetools>=5.5.0",
"cryptography>=42.0.0",
"impit>=0.6.1",
Expand Down
Loading
Loading