Skip to content

Make integrations tests more stable #301

@vdusek

Description

@vdusek

Our integration tests are highly unstable, often requiring multiple attempts to pass. Below are the two main issues we are encountering...

Issues during Actor creation

The first problem occurs when creating an Actor in conftest/make_actor (conftest.py#L247):

    ...
    build = await actor_client.build(version_number='0.0', wait_for_finish=300)
    assert build['status'] == ActorJobStatus.SUCCEEDED

The test frequently fails with the following assertion error:

>       assert build['status'] == ActorJobStatus.SUCCEEDED
E       AssertionError: assert 'READY' == <ActorJobStat...: 'SUCCEEDED'>
E         
E         - SUCCEEDED
E         + READY

Issues during Actor execution

Another common issue arises when checking the Actor's run status in the integration tests:

    ...
    run_result = await actor.call()
    assert run_result is not None
    assert run_result['status'] == 'SUCCEEDED'

Here, the test often fails with a similar mismatch in the expected status:

>       assert build['status'] == ActorJobStatus.SUCCEEDED
E       AssertionError: assert 'READY' == <ActorJobStat...: 'SUCCEEDED'>
E         
E         - SUCCEEDED
E         + READY

Metadata

Metadata

Assignees

Labels

infrastructureIssues related to infrastructure or CI/CD pipeline.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions