done

Author: vdusek

tests/integration/README.md

@@ -1,22 +1,16 @@
-Integration tests
-=================
+# Integration tests
 
-We have integration tests which build and run Actors using the Python SDK on the Apify Platform.
-To run these tests, you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to use for the tests,
-and then start them with `make integration-tests`.
+We have integration tests which build and run Actors using the Python SDK on the Apify Platform. To run these tests, you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to use for the tests, and then start them with `make integration-tests`.
 
-If you want to run the integration tests on a different environment than the main Apify Platform,
-you need to set the `APIFY_INTEGRATION_TESTS_API_URL` environment variable to the right URL to the Apify API you want to use.
+If you want to run the integration tests on a different environment than the main Apify Platform, you need to set the `APIFY_INTEGRATION_TESTS_API_URL` environment variable to the right URL to the Apify API you want to use.
 
-How to write tests
-------------------
+## How to write tests
 
 There are two fixtures which you can use to write tests:
 
 ### `apify_client_async`
 
-This fixture just gives you an instance of `ApifyClientAsync` configured with the right token and API URL,
-so you don't have to do that yourself.
+This fixture just gives you an instance of `ApifyClientAsync` configured with the right token and API URL, so you don't have to do that yourself.
 
 ```python
 async def test_something(apify_client_async: ApifyClientAsync) -> None:
@@ -27,64 +21,62 @@ async def test_something(apify_client_async: ApifyClientAsync) -> None:
 
 This fixture returns a factory function for creating Actors on the Apify Platform.
 
-For the Actor source, the fixture takes the files from `tests/integration/actor_source_base`,
-builds the Apify SDK wheel from the current codebase,
-and adds the Actor source you passed to the fixture as an argument.
-You have to pass exactly one of the `main_func`, `main_py` and `source_files` arguments.
+For the Actor source, the fixture takes the files from `tests/integration/actor_source_base`, builds the Apify SDK wheel from the current codebase, and adds the Actor source you passed to the fixture as an argument. You have to pass exactly one of the `main_func`, `main_py` and `source_files` arguments.
 
-The created Actor will be uploaded to the platform, built there, and after the test finishes, it will be automatically deleted.
-If the Actor build fails, it will not be deleted, so that you can check why the build failed.
+The created Actor will be uploaded to the platform, built there, and after the test finishes, it will be automatically deleted. If the Actor build fails, it will not be deleted, so that you can check why the build failed.
 
 ### Creating test Actor straight from a Python function
 
-You can create Actors straight from a Python function.
-This is great because you can have the test Actor source code checked with the linter.
+You can create Actors straight from a Python function. This is great because you can have the test Actor source code checked with the linter.
 
 ```python
-async def test_something(self, make_actor: ActorFactory) -> None:
+async def test_something(
+    make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
+) -> None:
     async def main() -> None:
         async with Actor:
             print('Hello!')
 
-    actor = await make_actor('something', main_func=main)
+    actor = await make_actor(label='something', main_func=main)
+    run_result = await run_actor(actor)
 
-    run_result = await actor.call()
-
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 ```
 
-These Actors will have the `src/main.py` file set to the `main` function definition,
-prepended with `import asyncio` and `from apify import Actor`, for your convenience.
+These Actors will have the `src/main.py` file set to the `main` function definition, prepended with `import asyncio` and `from apify import Actor`, for your convenience.
 
 You can also pass extra imports directly to the main function:
 
 ```python
-async def test_something(self, make_actor: ActorFactory) -> None:
+async def test_something(
+    make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
+) -> None:
     async def main():
         import os
         from apify_shared.consts import ActorEventTypes, ActorEnvVars
         async with Actor:
             print('The Actor is running with ' + os.getenv(ActorEnvVars.MEMORY_MBYTES) + 'MB of memory')
             await Actor.on(ActorEventTypes.SYSTEM_INFO, lambda event_data: print(event_data))
 
-    actor = await make_actor('something', main_func=main)
-
-    run_result = await actor.call()
+    actor = await make_actor(label='something', main_func=main)
+    run_result = await run_actor(actor)
 
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 ```
 
 ### Creating Actor from source files
 
-You can also pass the source files directly if you need something more complex
-(e.g. pass some fixed value to the Actor source code or use multiple source files).
+You can also pass the source files directly if you need something more complex (e.g. pass some fixed value to the Actor source code or use multiple source files).
 
 To pass the source code of the `src/main.py` file directly, use the `main_py` argument to `make_actor`:
 
 ```python
-async def test_something(self, make_actor: ActorFactory) -> None:
+async def test_something(
+    make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
+) -> None:
     expected_output = f'ACTOR_OUTPUT_{crypto_random_object_id(5)}'
     main_py_source = f"""
         import asyncio
@@ -96,21 +88,22 @@ async def test_something(self, make_actor: ActorFactory) -> None:
                 await Actor.set_value('OUTPUT', '{expected_output}')
     """
 
-    actor = await make_actor('something', main_py=main_py_source)
-
-    await actor.call()
+    actor = await make_actor(label='something', main_py=main_py_source)
+    await run_actor(actor)
 
     output_record = await actor.last_run().key_value_store().get_record('OUTPUT')
     assert output_record is not None
     assert output_record['value'] == expected_output
 
 ```
 
-Or you can pass multiple source files with the `source_files` argument,
-if you need something really complex:
+Or you can pass multiple source files with the `source_files` argument, if you need something really complex:
 
 ```python
-async def test_something(self, make_actor: ActorFactory) -> None:
+async def test_something(
+    make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
+) -> None:
     actor_source_files = {
         'src/utils.py': """
             from datetime import datetime, timezone
@@ -129,9 +122,8 @@ async def test_something(self, make_actor: ActorFactory) -> None:
                     print('Hello! It is ' + current_datetime.time())
         """,
     }
-    actor = await make_actor('something', source_files=actor_source_files)
+    actor = await make_actor(label='something', source_files=actor_source_files)
+    actor_run = await run_actor(actor)
 
-    actor_run = await actor.call()
-    assert actor_run is not None
-    assert actor_run['status'] == 'SUCCEEDED'
+    assert actor_run.status == 'SUCCEEDED'
 ```

tests/integration/conftest.py

@@ -24,9 +24,9 @@
 
     from apify_client.clients.resource_clients import ActorClientAsync
 
-TOKEN_ENV_VAR = 'APIFY_TEST_USER_API_TOKEN'
-API_URL_ENV_VAR = 'APIFY_INTEGRATION_TESTS_API_URL'
-SDK_ROOT_PATH = Path(__file__).parent.parent.parent.resolve()
+_TOKEN_ENV_VAR = 'APIFY_TEST_USER_API_TOKEN'
+_API_URL_ENV_VAR = 'APIFY_INTEGRATION_TESTS_API_URL'
+_SDK_ROOT_PATH = Path(__file__).parent.parent.parent.resolve()
 
 
 @pytest.fixture(autouse=True)
@@ -52,11 +52,11 @@ def apify_client_async() -> ApifyClientAsync:
     because `httpx.AsyncClient` in `ApifyClientAsync` tries to reuse the same event loop across requests,
     but `pytest-asyncio` closes the event loop after each test, and uses a new one for the next test.
     """
-    api_token = os.getenv(TOKEN_ENV_VAR)
-    api_url = os.getenv(API_URL_ENV_VAR)
+    api_token = os.getenv(_TOKEN_ENV_VAR)
+    api_url = os.getenv(_API_URL_ENV_VAR)
 
     if not api_token:
-        raise RuntimeError(f'{TOKEN_ENV_VAR} environment variable is missing, cannot run tests!')
+        raise RuntimeError(f'{_TOKEN_ENV_VAR} environment variable is missing, cannot run tests!')
 
     return ApifyClientAsync(api_token, api_url=api_url)
 
@@ -73,15 +73,15 @@ def sdk_wheel_path(tmp_path_factory: pytest.TempPathFactory, testrun_uid: str) -
         if not was_wheel_built_this_test_run_file.exists():
             subprocess.run(
                 args='python -m build',
-                cwd=SDK_ROOT_PATH,
+                cwd=_SDK_ROOT_PATH,
                 shell=True,
                 check=True,
                 capture_output=True,
             )
             was_wheel_built_this_test_run_file.touch()
 
         # Read the current package version, necessary for getting the right wheel filename.
-        pyproject_toml_file = (SDK_ROOT_PATH / 'pyproject.toml').read_text(encoding='utf-8')
+        pyproject_toml_file = (_SDK_ROOT_PATH / 'pyproject.toml').read_text(encoding='utf-8')
         for line in pyproject_toml_file.splitlines():
             if line.startswith('version = '):
                 delim = '"' if '"' in line else "'"
@@ -90,7 +90,7 @@ def sdk_wheel_path(tmp_path_factory: pytest.TempPathFactory, testrun_uid: str) -
         else:
             raise RuntimeError('Unable to find version string.')
 
-        wheel_path = SDK_ROOT_PATH / 'dist' / f'apify-{sdk_version}-py3-none-any.whl'
+        wheel_path = _SDK_ROOT_PATH / 'dist' / f'apify-{sdk_version}-py3-none-any.whl'
 
         # Just to be sure.
         assert wheel_path.exists()

tests/integration/test_actor_create_proxy_configuration.py

@@ -28,7 +28,7 @@ async def main() -> None:
             assert proxy_configuration._country_code == country_code
 
     actor = await make_actor(label='proxy-configuration', main_func=main)
-    run_result = await run_actor(actor=actor)
+    run_result = await run_actor(actor)
 
     assert run_result.status == 'SUCCEEDED'
 
@@ -65,6 +65,6 @@ async def main() -> None:
         await Actor.exit()
 
     actor = await make_actor(label='proxy-configuration', main_func=main)
-    run_result = await run_actor(actor=actor)
+    run_result = await run_actor(actor)
 
     assert run_result.status == 'SUCCEEDED'

tests/integration/test_actor_dataset.py

@@ -12,12 +12,12 @@
 
     from apify_client import ApifyClientAsync
 
-    from .conftest import MakeActorFunction
+    from .conftest import MakeActorFunction, RunActorFunction
 
 
 async def test_push_and_verify_data_in_default_dataset(
-    apify_client_async: ApifyClientAsync,
     make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
 ) -> None:
     desired_item_count = 100  # Also change inside main() if you're changing this
 
@@ -26,16 +26,10 @@ async def main() -> None:
         async with Actor:
             await Actor.push_data([{'id': i} for i in range(desired_item_count)])
 
-    actor = await make_actor('push-data', main_func=main)
-
-    call_result = await actor.call()
-    assert call_result is not None
-
-    run_client = apify_client_async.run(call_result['id'])
-    run_result = await run_client.wait_for_finish(wait_secs=600)
+    actor = await make_actor(label='push-data', main_func=main)
+    run_result = await run_actor(actor)
 
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 
     list_page = await actor.last_run().dataset().list_items()
     assert list_page.items[0]['id'] == 0
@@ -44,53 +38,41 @@ async def main() -> None:
 
 
 async def test_push_large_data_chunks_over_9mb(
-    apify_client_async: ApifyClientAsync,
     make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
 ) -> None:
     async def main() -> None:
         async with Actor:
             await Actor.push_data([{'str': 'x' * 10000} for _ in range(5000)])  # ~50MB
 
-    actor = await make_actor('push-data-over-9mb', main_func=main)
-
-    call_result = await actor.call()
-    assert call_result is not None
-
-    run_client = apify_client_async.run(call_result['id'])
-    run_result = await run_client.wait_for_finish(wait_secs=600)
+    actor = await make_actor(label='push-data-over-9mb', main_func=main)
+    run_result = await run_actor(actor)
 
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 
     async for item in actor.last_run().dataset().iterate_items():
         assert item['str'] == 'x' * 10000
 
 
 async def test_same_references_in_default_dataset(
-    apify_client_async: ApifyClientAsync,
     make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
 ) -> None:
     async def main() -> None:
         async with Actor:
             dataset1 = await Actor.open_dataset()
             dataset2 = await Actor.open_dataset()
             assert dataset1 is dataset2
 
-    actor = await make_actor('dataset-same-ref-default', main_func=main)
-
-    call_result = await actor.call()
-    assert call_result is not None
-
-    run_client = apify_client_async.run(call_result['id'])
-    run_result = await run_client.wait_for_finish(wait_secs=600)
+    actor = await make_actor(label='dataset-same-ref-default', main_func=main)
+    run_result = await run_actor(actor)
 
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 
 
 async def test_same_references_in_named_dataset(
-    apify_client_async: ApifyClientAsync,
     make_actor: MakeActorFunction,
+    run_actor: RunActorFunction,
 ) -> None:
     dataset_name = generate_unique_resource_name('dataset')
 
@@ -109,16 +91,10 @@ async def main() -> None:
 
             await dataset_by_name_1.drop()
 
-    actor = await make_actor('dataset-same-ref-named', main_func=main)
-
-    call_result = await actor.call(run_input={'datasetName': dataset_name})
-    assert call_result is not None
-
-    run_client = apify_client_async.run(call_result['id'])
-    run_result = await run_client.wait_for_finish(wait_secs=600)
+    actor = await make_actor(label='dataset-same-ref-named', main_func=main)
+    run_result = await run_actor(actor, run_input={'datasetName': dataset_name})
 
-    assert run_result is not None
-    assert run_result['status'] == 'SUCCEEDED'
+    assert run_result.status == 'SUCCEEDED'
 
 
 async def test_force_cloud(